Stratifying risk for dementia onset using large-scale electronic health record data: A retrospective cohort study

Reasoning language models for more transparent prediction of suicide risk

BACKGROUND

We previously demonstrated that a large language model could estimate suicide risk using hospital discharge notes.

OBJECTIVE

With the emergence of reasoning models that can be run on consumer-grade hardware, we investigated whether these models can approximate the performance of much larger and costlier models.

METHODS

From 458 053 adults hospitalised at one of two academic medical centres between 4 January 2005 and 2 January 2014, we identified 1995 who died by suicide or accident, and matched them with 5 control individuals. We used Llama-DeepSeek-R1 8B to generate predictions of risk. Beyond discrimination and calibration, we examined the aspects of model reasoning-that is, the topics in the chain of thought-associated with correct or incorrect predictions.

FINDINGS

The cohort included 1995 individuals who died by suicide or accidental death and 9975 individuals matched 5:1, totalling 11 954 discharges and 58 933 person-years of follow-up. In Fine and Grey regression, hazard as estimated by the Llama3-distilled model was significantly associated with observed risk (unadjusted HR 4.65 (3.58-6.04)). The corresponding c-statistic was 0.64 (0.63-0.65), modestly poorer than the GPT4o model (0.67 (0.66-0.68)). In chain-of-thought reasoning, topics including Substance Abuse, Surgical Procedure, and Age-related Comorbidities were associated with correct predictions, while Fall-related Injury was associated with incorrect prediction.

CONCLUSIONS

Application of a reasoning model using local, consumer-grade hardware only modestly diminished performance in stratifying suicide risk.

CLINICAL IMPLICATIONS

Smaller models can yield more secure, scalable and transparent risk prediction.

The relationship between dementia staging scales, cognitive-behavioral scales and functionality in patients with cognitive impairment

INTRODUCTION

The aim of this study is to retrospectively evaluate the relationship between dementia stage, cognitive and behavioral scales, and functional status in patients with cognitive impairment.

METHODS

The medical records of patients over 50 years of age, who were followed up for cognitive impairment at the neurology outpatient clinic were retrospectively scanned between January 1990 and November 2022. The Clinical Dementia Rating (CDR), Global Deterioration Scale (GDS) and Functional Assessment Staging Test (FAST), The Mini Mental State Examination (MMSE) and the Alzheimer's Disease Assessment Scale-Cognitive Subscore (ADAS-Cog) were recorded. The neuropsychiatric symptoms were evaluated by the Neuropsychiatric Inventory (NPI) and The Behavioral Pathology in Alzheimer's Disease Rating Scale (BEHAVE-AD). Patients' Instrumental Assessment of Daily Living (IADL) scores were recorded to assess functional capacity.

RESULTS

This study analyzed 871 patients with cognitive impairment, 69.8% of whom were having functional impairment. Alzheimer's disease was the most common type of dementia (64.6%), with memory problems as the key symptom (65.6%). Neuropsychiatric symptoms such as hallucinations, delusion, and eating disturbances were significantly associated with disability (p < 0.001), while depression and anxiety were not. CDR scale was the strongest predictor of disability (OR: 4.9, AUC = 0.740), outperforming other dementia staging scales. Cognitive and behavioral scales like MMSE and NPI showed stronger correlations with functional impairment than with the dementia staging scales (-0.132 < rs < 0.472, p < 0.001 and -0.284 < rs < -0.357, p < 0.001, respectively).

CONCLUSION

Our study demonstrated that both cognitive status and behavioral symptoms are critical in determining the level of functional impairment in cognitive impairments, but their contributions differ in magnitude and focus. As well as cognitive decline, neuropsychiatric symptoms may also need targeted management to reduce their impact on functionality. We need a practical tool that can be used in all stages of dementia, that does not overlook the impact of neuropsychological symptoms, and that can assess ADL according to the needs of patients and carers.

Dual-stream algorithms for dementia detection: Harnessing structured and unstructured electronic health record data, a novel approach to prevalence estimation

INTRODUCTION

Identifying individuals with dementia is crucial for prevalence estimation and service planning, but reliable, scalable methods are lacking. We developed novel set algorithms using both structured and unstructured electronic health record (EHR) data, applying Diagnostic and Statistical Manual of Mental Disorders criteria for dementia case identification.

METHODS

Our cohort (n = 1082) included individuals aged ≥ 60 with dementia identified through specialist clinics and a comparison group without dementia. Clinicians from Australia and the United States informed predictor selection. We developed algorithms through a biostatistics stream for structured data and a natural language processing (NLP) stream for text, synthesizing results via logistic regression.

RESULTS

The final structured model retained 16 variables (area under the receiver operating characteristic curve [AUC] 0.853, specificity 72.2%, sensitivity 80.6%). NLP classifiers (logistic regression, support vector machine, and random forest models) performed comparably. The final, combined model outperformed all others (AUC = 0.951, P < 0.001 for comparison to structured model).

DISCUSSION

Embedding text-derived insights within algorithms trained on structured medical data significantly enhances dementia identification capacity.

HIGHLIGHTS

Algorithmic tools for detection of individuals with dementia are available; however, previous work has used heterogeneous case definitions which are not clinically meaningful, and has relied on proxies such as diagnostic codes or medications for case ascertainment. We used a novel, dual-stream algorithmic development approach, simultaneously and separately modeling a clinically meaningful outcome (diagnosis of dementia according to specialized clinical impression) using structured and unstructured electronic health record datasets. Our clinically grounded case definition supported the inclusion of key structured variables (such as dementia International Classification of Disease codes and medications) as modeling predictors rather than outcomes. Our algorithms, published in detail to support validation and replication, represent a major step forward in the use of routinely collected data for detection of diagnosed dementia.

Detection Bias in EHR-Based Research on Clinical Exposures and Dementia

Importance

Detection bias occurs when an exposure is associated with a systematic difference in outcome ascertainment or diagnosis. For dementia research, diagnosed health conditions that bring patients into frequent interaction with health care may increase the chance that an individual receives a dementia diagnosis.

Objective

To evaluate potential detection bias or misdiagnosis bias in evaluation of clinical factors associated with dementia using electronic health record (EHR) data.

Design, Setting, and Participants

This prospective cohort study used data from 2 population-based volunteer cohorts: UK Biobank (UKB) and All of Us (AOU). Participants were aged 55 years or older, were dementia-free at baseline, and had linked EHRs. Participants in UKB were followed up from baseline (2006-2010) until December 2022, and in AOU, from baseline (2017-2022) until July 2022. Data were analyzed from November 2023 through February 2025.

Exposures

Diagnoses of type 2 diabetes, depression, hypertension, urinary tract infection, kidney stones, forearm fracture, and gastrointestinal (GI) bleeding.

Main Outcomes and Measures

Rate of incident all-cause dementia diagnosis from EHRs and associations between clinical exposures and incident dementia diagnosis, assessed using Cox proportional hazards regression models.

Results

Among 228 392 participants from UKB (n = 137 374; mean [SD] age at baseline, 62.5 [4.1] years; 53.8% female) and AOU (n = 91 018; mean [SD] age at baseline, 66.9 [7.8] years; 57.1% female), those with a history of a clinical exposure at baseline had higher dementia incidence rates compared with those without such history. For example, among participants with a history of GI bleeding, the dementia incidence rates were 3.0 (UKB) and 7.7 (AOU) per 1000 person-years compared with 2.2 (UKB) and 2.4 (AOU) per 1000 person-years among those without a history of GI bleeding. All exposures were significantly associated with incident dementia, with hazard ratios (HRs) ranging from 1.18 (95% CI, 1.00-1.40) to 3.51 (95% CI, 3.08-4.01). Risk of incident dementia was typically highest in the first year following exposure diagnosis and attenuated thereafter. For example, in the first year after GI bleeding, there were larger elevations in risk of incident dementia (HR, 2.17 [95% CI, 1.46-3.22] in UKB; HR, 2.56 [95% CI, 1.62-4.04] in AOU) compared with 1 to 5 years after bleeding (HR, 1.46 [95% CI, 1.15-1.86] in UKB; HR, 2.14 [95% CI, 1.63-2.81] in AOU).

Conclusions and Relevance

In this cohort study of 2 large datasets, diagnoses of several conditions associated with varying risks of dementia were associated with a higher short-term likelihood of dementia diagnosis. This finding suggests that diagnostic bias or misdiagnoses may lead to spurious associations between conditions requiring clinical care and subsequent dementia diagnoses.

Natural language processing of electronic health records for early detection of cognitive decline: a systematic review

This systematic review evaluated natural language processing (NLP) approaches for detecting cognitive impairment in electronic health record clinical notes. Following PRISMA guidelines, we analyzed 18 studies (n = 1,064,530) that employed rule-based algorithms (67%), traditional machine learning (28%), and deep learning (17%). NLP models demonstrated robust performance in identifying cognitive decline, with median sensitivity 0.88 (IQR 0.74-0.91) and specificity 0.96 (IQR 0.81-0.99). Deep learning architectures achieved superior results, with area under the receiver operating characteristic curves up to 0.997. Major implementation challenges included incomplete electronic health record data capture, inconsistent clinical documentation practices, and limited external validation. While NLP demonstrates promise, successful clinical translation requires establishing standardized approaches, improving access to annotated datasets, and developing equitable deployment frameworks.

Advancing Mental Health Research Through Strategic Integration of Transdiagnostic Dimensions and Genomics

Genome-wide studies are yielding a growing catalog of common and rare variants that confer risk for psychopathology. However, despite representing unprecedented progress, emerging data also indicate that the full promise of psychiatric genetics-including understanding pathophysiology and improving personalized care-will not be fully realized by targeting traditional dichotomous diagnostic categories. The current article provides reflections on themes that emerged from a 2021 National Institute of Mental Health-sponsored conference convened to address strategies for the evolving field of psychiatric genetics. As anticipated by the National Institute of Mental Health's Research Domain Criteria framework, multilevel investigations of dimensional and transdiagnostic phenotypes, particularly when integrated with biobanks and big data, will be critical to advancing knowledge. The path forward will also require more diverse representation in source studies. Additionally, progress will be catalyzed by a range of converging approaches, including capitalizing on computational methods, pursuing biological insights, working within a developmental framework, and engaging health care systems and patient communities.

Real-World Insights Into Dementia Diagnosis Trajectory and Clinical Practice Patterns Unveiled by Natural Language Processing: Development and Usability Study

Background

Understanding the dementia disease trajectory and clinical practice patterns in outpatient settings is vital for effective management. Knowledge about the path from initial memory loss complaints to dementia diagnosis remains limited.

Objective

This study aims to (1) determine the time intervals between initial memory loss complaints and dementia diagnosis in outpatient care, (2) assess the proportion of patients receiving cognition-enhancing medication prior to dementia diagnosis, and (3) identify patient and provider characteristics that influence the time between memory complaints and diagnosis and the prescription of cognition-enhancing medication.

Methods

This retrospective cohort study used a large outpatient electronic health record (EHR) database from the University of Connecticut Health Center, covering 2010-2018, with a cohort of 581 outpatients. We used a customized deep learning-based natural language processing (NLP) pipeline to extract clinical information from EHR data, focusing on cognition-related symptoms, primary caregiver relation, and medication usage. We applied descriptive statistics, linear, and logistic regression for analysis.

Results

The NLP pipeline showed precision, recall, and F1-scores of 0.97, 0.93, and 0.95, respectively. The median time from the first memory loss complaint to dementia diagnosis was 342 (IQR 200-675) days. Factors such as the location of initial complaints and diagnosis and primary caregiver relationships significantly affected this interval. Around 25.1% (146/581) of patients were prescribed cognition-enhancing medication before diagnosis, with the number of complaints influencing medication usage.

Conclusions

Our NLP-guided analysis provided insights into the clinical pathways from memory complaints to dementia diagnosis and medication practices, which can enhance patient care and decision-making in outpatient settings.

Machine learning prediction algorithms for 2- , 5- and 10-year risk of Alzheimer's, Parkinson's and dementia at age 65: a study using medical records from France and the UK General Practitioners

Background

Leveraging machine learning on electronic health records offers a promising method for early identification of individuals at risk for dementia and neurodegenerative diseases. Current risk algorithms heavily rely on age, highlighting the need for alternative models with strong predictive power, especially at age 65, a crucial time for early screening and prevention.

Methods

This prospective study analyzed electronic health records (EHR) from 76,427 adults (age 65, 52.1% women) using the THIN database. A general risk algorithm for Alzheimer's disease, Parkinson's disease, and dementia was developed using machine learning to select predictors from diagnoses, and medications.

Results

Medications (e.g., laxatives, urological drugs, antidepressants), along with sex, BMI, and comorbidities, were key predictors. The algorithm achieved a 38.4% detection rate at a 5% false-positive rate for 2-year dementia prediction.

Conclusion

The validated prediction algorithms, easy to implement in primary care, identify high-risk 65-year-olds using medication records. Further refinement and broader validation are needed.

Disease-Specific Risk Models for Predicting Dementia: An Umbrella Review

Dementia is a leading cause of disability and death globally. Individuals with diseases such as cardiovascular, cardiometabolic and cerebrovascular disease are often at increased dementia risk. However, while numerous models have been developed to predict dementia, they are often not tailored to disease-specific groups. Yet, different disease groups may have unique risk factor profiles and tailored models that account for these differences may have enhanced predictive accuracy. In this review, we synthesise findings from three previous systematic reviews on dementia risk model development and testing to present an overview of the literature on dementia risk prediction modelling in people with a history of disease. Nine studies met the inclusion criteria. Currently, disease-specific models have only been developed in people with a history of diabetes where demographic, disease-specific and comorbidity data were used. Some existing risk models, including CHA2DS2-VASc and CHADS2, have been externally validated for dementia outcomes in those with atrial fibrillation and heart failure. One study developed a dementia risk model for their whole population, which had similar predictive accuracy when applied in a sub-sample with stroke. This emphasises the importance of considering disease status in identifying key predictors for dementia and generating accurate prediction models for dementia.

Risk factors and risk stratification approaches for delirium screening: A Geriatric Emergency Department Guidelines 2.0 systematic review

OBJECTIVE

As part of the Geriatric Emergency Department (ED) Guidelines 2.0 project, we conducted a systematic review to find risk factors or risk stratification approaches that can be used to identify subsets of older adults who may benefit from targeted ED delirium screening.

METHODS

An electronic search strategy was developed with a medical librarian, conducted in April 2021 and November 2022. Full-text studies of patients ≥65 years assessed for prevalent delirium in the ED were included. Risk of bias was assessed using the McMaster University Clarity Group tool. Outcomes measures pertained to the risk stratification method used. Due to heterogeneity of patient populations, risk stratification methods, and outcomes, a meta-analysis was not conducted.

RESULTS

Our search yielded 1878 unique citations, of which 13 were included. Six studies developed a novel delirium risk score with or without evaluation of specific risk factors, six studies evaluated specific risk factors only, and one study evaluated an existing nondelirium risk score for association with delirium. The most common risk factor was history of dementia, with odds ratios ranging from 3.3 (95% confidence interval [CI] 1.2-8.9) to 18.33 (95% CI 8.08-43.64). Other risk factors that were consistently associated with increased risk of delirium included older age, use of certain medications (such as antipsychotics, antidepressants, and opioids, among others), and functional impairments. Of the studies that developed novel risk scores, the reported area under the curve ranged from 0.77 to 0.90. Only two studies reported potential impact of the risk stratification tool on screening burden.

CONCLUSIONS

There is significant heterogeneity, but results suggest that factors such as dementia, age over 75, and functional impairments should be used to identify older adults who are at highest risk for ED delirium. No studies evaluated implementation of a risk stratification method for delirium screening or evaluated patient-oriented outcomes.

Extracting Critical Information from Unstructured Clinicians' Notes Data to Identify Dementia Severity Using a Rule-Based Approach: Feasibility Study

BACKGROUND

The severity of Alzheimer disease and related dementias (ADRD) is rarely documented in structured data fields in electronic health records (EHRs). Although this information is important for clinical monitoring and decision-making, it is often undocumented or "hidden" in unstructured text fields and not readily available for clinicians to act upon.

OBJECTIVE

We aimed to assess the feasibility and potential bias in using keywords and rule-based matching for obtaining information about the severity of ADRD from EHR data.

METHODS

We used EHR data from a large academic health care system that included patients with a primary discharge diagnosis of ADRD based on ICD-9 (International Classification of Diseases, Ninth Revision) and ICD-10 (International Statistical Classification of Diseases, Tenth Revision) codes between 2014 and 2019. We first assessed the presence of ADRD severity information and then the severity of ADRD in the EHR. Clinicians' notes were used to determine the severity of ADRD based on two criteria: (1) scores from the Mini Mental State Examination and Montreal Cognitive Assessment and (2) explicit terms for ADRD severity (eg, "mild dementia" and "advanced Alzheimer disease"). We compiled a list of common ADRD symptoms, cognitive test names, and disease severity terms, refining it iteratively based on previous literature and clinical expertise. Subsequently, we used rule-based matching in Python using standard open-source data analysis libraries to identify the context in which specific words or phrases were mentioned. We estimated the prevalence of documented ADRD severity and assessed the performance of our rule-based algorithm.

RESULTS

We included 9115 eligible patients with over 65,000 notes from the providers. Overall, 22.93% (2090/9115) of patients were documented with mild ADRD, 20.87% (1902/9115) were documented with moderate or severe ADRD, and 56.20% (5123/9115) did not have any documentation of the severity of their ADRD. For the task of determining the presence of any ADRD severity information, our algorithm achieved an accuracy of >95%, specificity of >95%, sensitivity of >90%, and an F1-score of >83%. For the specific task of identifying the actual severity of ADRD, the algorithm performed well with an accuracy of >91%, specificity of >80%, sensitivity of >88%, and F1-score of >92%. Comparing patients with mild ADRD to those with more advanced ADRD, the latter group tended to contain older, more likely female, and Black patients, and having received their diagnoses in primary care or in-hospital settings. Relative to patients with undocumented ADRD severity, those with documented ADRD severity had a similar distribution in terms of sex, race, and rural or urban residence.

CONCLUSIONS

Our study demonstrates the feasibility of using a rule-based matching algorithm to identify ADRD severity from unstructured EHR report data. However, it is essential to acknowledge potential biases arising from differences in documentation practices across various health care systems.

What's New in Dementia Risk Prediction Modelling? An Updated Systematic Review

Introduction

Identifying individuals at high risk of dementia is critical to optimized clinical care, formulating effective preventative strategies, and determining eligibility for clinical trials. Since our previous systematic reviews in 2010 and 2015, there has been a surge in dementia risk prediction modelling. The aim of this study was to update our previous reviews to explore, and critically review, new developments in dementia risk modelling.

Methods

MEDLINE, Embase, Scopus, and Web of Science were searched from March 2014 to June 2022. Studies were included if they were population- or community-based cohorts (including electronic health record data), had developed a model for predicting late-life incident dementia, and included model performance indices such as discrimination, calibration, or external validation.

Results

In total, 9,209 articles were identified from the electronic search, of which 74 met the inclusion criteria. We found a substantial increase in the number of new models published from 2014 (>50 new models), including an increase in the number of models developed using machine learning. Over 450 unique predictor (component) variables have been tested. Nineteen studies (26%) undertook external validation of newly developed or existing models, with mixed results. For the first time, models have also been developed in low- and middle-income countries (LMICs) and others validated in racial and ethnic minority groups.

Conclusion

The literature on dementia risk prediction modelling is rapidly evolving with new analytical developments and testing in LMICs. However, it is still challenging to make recommendations about which one model is the most suitable for routine use in a clinical setting. There is an urgent need to develop a suitable, robust, validated risk prediction model in the general population that can be widely implemented in clinical practice to improve dementia prevention.

Natural language processing with machine learning methods to analyze unstructured patient-reported outcomes derived from electronic health records: A systematic review

OBJECTIVE

Natural language processing (NLP) combined with machine learning (ML) techniques are increasingly used to process unstructured/free-text patient-reported outcome (PRO) data available in electronic health records (EHRs). This systematic review summarizes the literature reporting NLP/ML systems/toolkits for analyzing PROs in clinical narratives of EHRs and discusses the future directions for the application of this modality in clinical care.

METHODS

We searched PubMed, Scopus, and Web of Science for studies written in English between 1/1/2000 and 12/31/2020. Seventy-nine studies meeting the eligibility criteria were included. We abstracted and summarized information related to the study purpose, patient population, type/source/amount of unstructured PRO data, linguistic features, and NLP systems/toolkits for processing unstructured PROs in EHRs.

RESULTS

Most of the studies used NLP/ML techniques to extract PROs from clinical narratives (n = 74) and mapped the extracted PROs into specific PRO domains for phenotyping or clustering purposes (n = 26). Some studies used NLP/ML to process PROs for predicting disease progression or onset of adverse events (n = 22) or developing/validating NLP/ML pipelines for analyzing unstructured PROs (n = 19). Studies used different linguistic features, including lexical, syntactic, semantic, and contextual features, to process unstructured PROs. Among the 25 NLP systems/toolkits we identified, 15 used rule-based NLP, 6 used hybrid NLP, and 4 used non-neural ML algorithms embedded in NLP.

CONCLUSIONS

This study supports the potential utility of different NLP/ML techniques in processing unstructured PROs available in EHRs for clinical care. Though using annotation rules for NLP/ML to analyze unstructured PROs is dominant, deploying novel neural ML-based methods is warranted.

Passive digital markers for Alzheimer's disease and other related dementias: A systematic evidence review

BACKGROUND

The timely detection of Alzheimer's disease and other related dementias (ADRD) is suboptimal. Digital data already stored in electronic health records (EHR) offer opportunities for enhancing the timely detection of ADRD by facilitating the development of passive digital markers (PDMs). We conducted a systematic evidence review to identify studies that describe the development, performance, and validity of EHR-based PDMs for ADRD.

METHODS

We searched the literature published from January 2000 to August 2022 and reviewed cross-sectional, retrospective, or prospective observational studies with a patient population of 18 years or older, published in English that collected and interpreted original data, included EHR as a source of digital data, and had the primary purpose of supporting ADRD care. We extracted relevant data from the included studies with guidance from the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist and used the US Preventive Services Task Force criteria to appraise each study.

RESULTS

We included and appraised 19 studies. Four studies were considered to have a fair quality, and none was considered to have a good quality. The functionality of the PDMs varied from detecting mild cognitive impairment, Alzheimer's disease or ADRD, to forecasting stages of ADRD. Only seven studies used a valid reference diagnostic method. Nine PDMs used only structured EHR data, and five studies provided complete information on the race and ethnicity of its population. The number of features included in the PDMs ranges from 10 to 853, and the PMDs used a variety of statistical and machine learning algorithms with various time-at-risk windows. The area under the curve (AUC) for the PDMs varied from 0.67 to 0.97.

CONCLUSION

Although we noted heterogeneity in the PDMs development and performance, there is evidence that these PDMs have the potential to detect ADRD at earlier stages.

Dimensional clinical phenotyping using post-mortem brain donor medical records: post-mortem RDoC profiling is associated with Alzheimer's disease neuropathology

Introduction

Transdiagnostic dimensional phenotypes are essential to investigate the relationship between continuous symptom dimensions and pathological changes. This is a fundamental challenge to post-mortem work, as assessments of phenotypic concepts need to rely on existing records.

Methods

We adapted well-validated methodologies to compute National Institute of Mental Health Research Domain Criteria (RDoC) scores using natural language processing (NLP) from electronic health records (EHRs) obtained from post-mortem brain donors and tested whether cognitive domain scores were associated with Alzheimer's disease neuropathological measures.

Results

Our results confirm an association of EHR-derived cognitive scores with neuropathological findings. Notably, higher neuropathological load, particularly neuritic plaques, was associated with higher cognitive burden scores in the frontal (ß = 0.38, P = 0.0004), parietal (ß = 0.35, P = 0.0008), temporal (ß = 0.37, P = 0.0004) and occipital (ß = 0.37, P = 0.0003) lobes.

Discussion

This proof-of-concept study supports the validity of NLP-based methodologies to obtain quantitative measures of RDoC clinical domains from post-mortem EHR. The associations may accelerate post-mortem brain research beyond classical case-control designs.

Dementia and electronic health record phenotypes: a scoping review of available phenotypes and opportunities for future research

OBJECTIVE

We performed a scoping review of algorithms using electronic health record (EHR) data to identify patients with Alzheimer's disease and related dementias (ADRD), to advance their use in research and clinical care.

MATERIALS AND METHODS

Starting with a previous scoping review of EHR phenotypes, we performed a cumulative update (April 2020 through March 1, 2023) using Pubmed, PheKB, and expert review with exclusive focus on ADRD identification. We included algorithms using EHR data alone or in combination with non-EHR data and characterized whether they identified patients at high risk of or with a current diagnosis of ADRD.

RESULTS

For our cumulative focused update, we reviewed 271 titles meeting our search criteria, 49 abstracts, and 26 full text papers. We identified 8 articles from the original systematic review, 8 from our new search, and 4 recommended by an expert. We identified 20 papers describing 19 unique EHR phenotypes for ADRD: 7 algorithms identifying patients with diagnosed dementia and 12 algorithms identifying patients at high risk of dementia that prioritize sensitivity over specificity. Reference standards range from only using other EHR data to in-person cognitive screening.

CONCLUSION

A variety of EHR-based phenotypes are available for use in identifying populations with or at high-risk of developing ADRD. This review provides comparative detail to aid in choosing the best algorithm for research, clinical care, and population health projects based on the use case and available data. Future research may further improve the design and use of algorithms by considering EHR data provenance.

Ethical issues when using digital biomarkers and artificial intelligence for the early detection of dementia

Dementia poses a growing challenge for health services but remains stigmatized and under-recognized. Digital technologies to aid the earlier detection of dementia are approaching market. These include traditional cognitive screening tools presented on mobile devices, smartphone native applications, passive data collection from wearable, in-home and in-car sensors, as well as machine learning techniques applied to clinic and imaging data. It has been suggested that earlier detection and diagnosis may help patients plan for their future, achieve a better quality of life, and access clinical trials and possible future disease modifying treatments. In this review, we explore whether digital tools for the early detection of dementia can or should be deployed, by assessing them against the principles of ethical screening programs. We conclude that while the importance of dementia as a health problem is unquestionable, significant challenges remain. There is no available treatment which improves the prognosis of diagnosed disease. Progression from early-stage disease to dementia is neither given nor currently predictable. Available technologies are generally not both minimally invasive and highly accurate. Digital deployment risks exacerbating health inequalities due to biased training data and inequity in digital access. Finally, the acceptability of early dementia detection is not established, and resources would be needed to ensure follow-up and support for those flagged by any new system. We conclude that early dementia detection deployed at scale via digital technologies does not meet standards for a screening program and we offer recommendations for moving toward an ethical mode of implementation. This article is categorized under:Application Areas > Health CareCommercial, Legal, and Ethical Issues > Ethical ConsiderationsTechnologies > Artificial Intelligence.

Can Patients with Dementia Be Identified in Primary Care Electronic Medical Records Using Natural Language Processing?

Dementia and mild cognitive impairment can be underrecognized in primary care practice and research. Free-text fields in electronic medical records (EMRs) are a rich source of information which might support increased detection and enable a better understanding of populations at risk of dementia. We used natural language processing (NLP) to identify dementia-related features in EMRs and compared the performance of supervised machine learning models to classify patients with dementia. We assembled a cohort of primary care patients aged 66 + years in Ontario, Canada, from EMR notes collected until December 2016: 526 with dementia and 44,148 without dementia. We identified dementia-related features by applying published lists, clinician input, and NLP with word embeddings to free-text progress and consult notes and organized features into thematic groups. Using machine learning models, we compared the performance of features to detect dementia, overall and during time periods relative to dementia case ascertainment in health administrative databases. Over 900 dementia-related features were identified and grouped into eight themes (including symptoms, social, function, cognition). Using notes from all time periods, LASSO had the best performance (F1 score: 77.2%, sensitivity: 71.5%, specificity: 99.8%). Model performance was poor when notes written before case ascertainment were included (F1 score: 14.4%, sensitivity: 8.3%, specificity 99.9%) but improved as later notes were added. While similar models may eventually improve recognition of cognitive issues and dementia in primary care EMRs, our findings suggest that further research is needed to identify which additional EMR components might be useful to promote early detection of dementia.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00125-6.

Recommended practices and ethical considerations for natural language processing-assisted observational research: A scoping review

An increasing number of studies have reported using natural language processing (NLP) to assist observational research by extracting clinical information from electronic health records (EHRs). Currently, no standardized reporting guidelines for NLP-assisted observational studies exist. The absence of detailed reporting guidelines may create ambiguity in the use of NLP-derived content, knowledge gaps in the current research reporting practices, and reproducibility challenges. To address these issues, we conducted a scoping review of NLP-assisted observational clinical studies and examined their reporting practices, focusing on NLP methodology and evaluation. Through our investigation, we discovered a high variation regarding the reporting practices, such as inconsistent use of references for measurement studies, variation in the reporting location (reference, appendix, and manuscript), and different granularity of NLP methodology and evaluation details. To promote the wide adoption and utilization of NLP solutions in clinical research, we outline several perspectives that align with the six principles released by the World Health Organization (WHO) that guide the ethical use of artificial intelligence for health.

Longitudinal validation of an electronic health record delirium prediction model applied at admission in COVID-19 patients

OBJECTIVE

To validate a previously published machine learning model of delirium risk in hospitalized patients with coronavirus disease 2019 (COVID-19).

METHOD

Using data from six hospitals across two academic medical networks covering care occurring after initial model development, we calculated the predicted risk of delirium using a previously developed risk model applied to diagnostic, medication, laboratory, and other clinical features available in the electronic health record (EHR) at time of hospital admission. We evaluated the accuracy of these predictions against subsequent delirium diagnoses during that admission.

RESULTS

Of the 5102 patients in this cohort, 716 (14%) developed delirium. The model's risk predictions produced a c-index of 0.75 (95% CI, 0.73-0.77) with 27.7% of cases occurring in the top decile of predicted risk scores. Model calibration was diminished compared to the initial COVID-19 wave.

CONCLUSION

This EHR delirium risk prediction model, developed during the initial surge of COVID-19 patients, produced consistent discrimination over subsequent larger waves; however, with changing cohort composition and delirium occurrence rates, model calibration decreased. These results underscore the importance of calibration, and the challenge of developing risk models for clinical contexts where standard of care and clinical populations may shift.

Natural language processing in clinical neuroscience and psychiatry: A review

Natural language processing (NLP) is rapidly becoming an important topic in the medical community. The ability to automatically analyze any type of medical document could be the key factor to fully exploit the data it contains. Cutting-edge artificial intelligence (AI) architectures, particularly machine learning and deep learning, have begun to be applied to this topic and have yielded promising results. We conducted a literature search for 1,024 papers that used NLP technology in neuroscience and psychiatry from 2010 to early 2022. After a selection process, 115 papers were evaluated. Each publication was classified into one of three categories: information extraction, classification, and data inference. Automated understanding of clinical reports in electronic health records has the potential to improve healthcare delivery. Overall, the performance of NLP applications is high, with an average F1-score and AUC above 85%. We also derived a composite measure in the form of Z-scores to better compare the performance of NLP models and their different classes as a whole. No statistical differences were found in the unbiased comparison. Strong asymmetry between English and non-English models, difficulty in obtaining high-quality annotated data, and train biases causing low generalizability are the main limitations. This review suggests that NLP could be an effective tool to help clinicians gain insights from medical reports, clinical research forms, and more, making NLP an effective tool to improve the quality of healthcare services.

Characteristics of Dimensional Psychopathology in Suicidal Patients With Major Psychiatric Disorders and Its Association With the Length of Hospital Stay: Algorithm Validation Study

BACKGROUND

Suicide has emerged as a serious concern for public health; however, only few studies have revealed the differences between major psychiatric disorders and suicide. Recent studies have attempted to quantify research domain criteria (RDoC) into numeric scores to systematically use them in computerized methods. The RDoC scores were used to reveal the characteristics of suicide and its association with major psychiatric disorders.

OBJECTIVE

We intended to investigate the differences in the dimensional psychopathology among hospitalized suicidal patients and the association between the dimensional psychopathology of psychiatric disorders and length of hospital stay.

METHODS

This retrospective study enrolled hospitalized suicidal patients diagnosed with major psychiatric disorders (depression, schizophrenia, and bipolar disorder) between January 2010 and December 2020 at a tertiary hospital in South Korea. The RDoC scores were calculated using the patients' admission notes. To measure the differences between psychiatric disorder cohorts, analysis of variance and the Cochran Q test were conducted and post hoc analysis for RDoC domains was performed with the independent two-sample t test. A linear regression model was used to analyze the association between the RDoC scores and sociodemographic features and comorbidity index. To estimate the association between the RDoC scores and length of hospital stay, multiple logistic regression models were applied to each psychiatric disorder group.

RESULTS

We retrieved 732 admissions for 571 patients (465 with depression, 73 with schizophrenia, and 33 with bipolar disorder). We found significant differences in the dimensional psychopathology according to the psychiatric disorders. The patient group with depression showed the highest negative RDoC domain scores. In the cognitive and social RDoC domains, the groups with schizophrenia and bipolar disorder scored higher than the group with depression. In the arousal RDoC domain, the depression and bipolar disorder groups scored higher than the group with schizophrenia. We identified significant associations between the RDoC scores and length of stay for the depression and bipolar disorder groups. The odds ratios (ORs) of the length of stay were increased because of the higher negative RDoC domain scores in the group with depression (OR 1.058, 95% CI 1.006-1.114) and decreased by higher arousal RDoC domain scores in the group with bipolar disorder (OR 0.537, 95% CI 0.285-0.815).

CONCLUSIONS

This study showed the association between the dimensional psychopathology of major psychiatric disorders related to suicide and the length of hospital stay and identified differences in the dimensional psychopathology of major psychiatric disorders. This may provide new perspectives for understanding suicidal patients.

Identifying Medicare beneficiaries with dementia

BACKGROUND/OBJECTIVES

No data exist regarding the validity of International Classification of Disease (ICD)-10 dementia diagnoses against a clinician-adjudicated reference standard within Medicare claims data. We examined the accuracy of claims-based diagnoses with respect to expert clinician adjudication using a novel database with individual-level linkages between electronic health record (EHR) and claims.

DESIGN

In this retrospective observational study, two neurologists and two psychiatrists performed a standardized review of patients' medical records from January 2016 to December 2018 and adjudicated dementia status. We measured the accuracy of three claims-based definitions of dementia against the reference standard.

SETTING

Mass-General-Brigham Healthcare (MGB), Massachusetts, USA.

PARTICIPANTS

From an eligible population of 40,690 fee-for-service (FFS) Medicare beneficiaries, aged 65 years and older, within the MGB Accountable Care Organization (ACO), we generated a random sample of 1002 patients, stratified by the pretest likelihood of dementia using administrative surrogates.

INTERVENTION

None.

MEASUREMENTS

We evaluated the accuracy (area under receiver operating curve [AUROC]) and calibration (calibration-in-the-large [CITL] and calibration slope) of three ICD-10 claims-based definitions of dementia against clinician-adjudicated standards. We applied inverse probability weighting to reconstruct the eligible population and reported the mean and 95% confidence interval (95% CI) for all performance characteristics, using 10-fold cross-validation (CV).

RESULTS

Beneficiaries had an average age of 75.3 years and were predominately female (59%) and non-Hispanic whites (93%). The adjudicated prevalence of dementia in the eligible population was 7%. The best-performing definition demonstrated excellent accuracy (CV-AUC 0.94; 95% CI 0.92-0.96) and was well-calibrated to the reference standard of clinician-adjudicated dementia (CV-CITL <0.001, CV-slope 0.97).

CONCLUSION

This study is the first to validate ICD-10 diagnostic codes against a robust and replicable approach to dementia ascertainment, using a real-world clinical reference standard. The best performing definition includes diagnostic codes with strong face validity and outperforms an updated version of a previously validated ICD-9 definition of dementia.

Machine learning for modeling the progression of Alzheimer disease dementia using clinical data: a systematic literature review

OBJECTIVE

Alzheimer disease (AD) is the most common cause of dementia, a syndrome characterized by cognitive impairment severe enough to interfere with activities of daily life. We aimed to conduct a systematic literature review (SLR) of studies that applied machine learning (ML) methods to clinical data derived from electronic health records in order to model risk for progression of AD dementia.

MATERIALS AND METHODS

We searched for articles published between January 1, 2010, and May 31, 2020, in PubMed, Scopus, ScienceDirect, IEEE Explore Digital Library, Association for Computing Machinery Digital Library, and arXiv. We used predefined criteria to select relevant articles and summarized them according to key components of ML analysis such as data characteristics, computational algorithms, and research focus.

RESULTS

There has been a considerable rise over the past 5 years in the number of research papers using ML-based analysis for AD dementia modeling. We reviewed 64 relevant articles in our SLR. The results suggest that majority of existing research has focused on predicting progression of AD dementia using publicly available datasets containing both neuroimaging and clinical data (neurobehavioral status exam scores, patient demographics, neuroimaging data, and laboratory test values).

DISCUSSION

Identifying individuals at risk for progression of AD dementia could potentially help to personalize disease management to plan future care. Clinical data consisting of both structured data tables and clinical notes can be effectively used in ML-based approaches to model risk for AD dementia progression. Data sharing and reproducibility of results can enhance the impact, adaptation, and generalizability of this research.

Using Healthcare Data in Embedded Pragmatic Clinical Trials among People Living with Dementia and Their Caregivers: State of the Art

Embedded pragmatic clinical trials (ePCTs) are embedded in healthcare systems as well as their data environments. For people living with dementia (PLWD), settings of care can be different from the general population and involve additional people whose information is also important. The ePCT designs have the opportunity to leverage data that becomes available through the normal delivery of care. They may be particularly valuable in Alzheimer's disease and Alzheimer's disease-related dementia (AD/ADRD), given the complexity of case identification and the diversity of care settings. Grounded in the objectives of the Data and Technical Core of the newly established National Institute on Aging Imbedded Pragmatic Alzheimer's Disease and AD-Related Dementias Clinical Trials Collaboratory (IMPACT Collaboratory), this article summarizes the state of the art in using existing data sources (eg, Medicare claims, electronic health records) in AD/ADRD ePCTs and approaches to integrating them in real-world settings. J Am Geriatr Soc 68:S49-S54, 2020.

Mood Disorders and Outcomes of COVID-19 Hospitalizations

OBJECTIVE

The authors sought to characterize the association between prior mood disorder diagnosis and hospital outcomes among individuals admitted with COVID-19 to six Eastern Massachusetts hospitals.

METHODS

A retrospective cohort was drawn from the electronic health records of two academic medical centers and four community hospitals between February 15 and May 24, 2020. Associations between history of mood disorder and in-hospital mortality and hospital discharge home were examined using regression models among any hospitalized patients with positive tests for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).

RESULTS

Among 2,988 admitted individuals, 717 (24.0%) had a prior mood disorder diagnosis. In Cox regression models adjusted for age, sex, and hospital site, presence of a mood disorder prior to admission was associated with greater in-hospital mortality risk beyond hospital day 12 (crude hazard ratio=2.156, 95% CI=1.540, 3.020; fully adjusted hazard ratio=1.540, 95% CI=1.054, 2.250). A mood disorder diagnosis was also associated with greater likelihood of discharge to a skilled nursing facility or other rehabilitation facility rather than home (crude odds ratio=2.035, 95% CI=1.661, 2.493; fully adjusted odds ratio=1.504, 95% CI=1.132, 1.999).

CONCLUSIONS

Hospitalized individuals with a history of mood disorder may be at risk for greater COVID-19 morbidity and mortality and are at increased risk of need for postacute care. Further studies should investigate the mechanism by which these disorders may confer elevated risk.

Development and External Validation of a Delirium Prediction Model for Hospitalized Patients With Coronavirus Disease 2019

Background

The coronavirus disease 2019 pandemic has placed unprecedented stress on health systems and has been associated with elevated risk for delirium. The convergence of pandemic resource limitation and clinical demand associated with delirium requires careful risk stratification for targeted prevention efforts.

Objectives

To develop an incident delirium predictive model among coronavirus disease 2019 patients.

Methods

We applied supervised machine learning to electronic health record data for inpatients with coronavirus disease 2019 at three hospitals to build an incident delirium diagnosis prediction model. We validated this model in three different hospitals. Both hospital cohorts included academic and community settings.

Results

Among 2907 patients across 6 hospitals, 488 (16.8%) developed delirium. Applying the predictive model in the external validation cohort of 755 patients, the c-index was 0.75 (0.71–0.79) and the lift in the top quintile was 2.1. At a sensitivity of 80%, the specificity was 56%, negative predictive value 92%, and positive predictive value 30%. Equivalent model performance was observed in subsamples stratified by age, sex, race, need for critical care and care at community vs. academic hospitals.

Conclusion

Machine learning applied to electronic health records available at the time of inpatient admission can be used to risk-stratify patients with coronavirus disease 2019 for incident delirium. Delirium is common among patients with coronavirus disease 2019, and resource constraints during a pandemic demand careful attention to the optimal application of predictive models.

Mapping of Transdiagnostic Neuropsychiatric Phenotypes Across Patients in Two General Hospitals

BACKGROUND

Multidimensional transdiagnostic phenotyping systems are increasingly important to neuropsychiatric phenotyping, particularly in translational research settings. The relationship the National Institute of Mental Health's Research Domain Criteria multidimensional approach to psychopathology and nonpsychiatric diagnoses has not been studied at scale but is relevant to those caring for neuropsychiatric illness in medical and surgical settings.

METHODS

We applied the CQH Dimensional Phenotyper natural language processing tool to estimate National Institute of Mental Health's Research Domain Criteria domain-associated symptoms of individuals admitted to nonpsychiatric wards at each of 2 large academic general hospitals over an 8-year period. We compared patterns in individual domain symptom burden, as well as a new pooled unidimensional measure, by primary medical and surgical diagnosis.

RESULTS

Analysis included 227,243 patients from hospital 1 of whom 68,793 (30.3%) had a prior psychiatric history and 220,213 patients from hospital 2 of whom 50,818 (23.1%) had a prior psychiatric history. The distribution of Research Domain Criteria symptom burdens over primary diagnosis was similar across hospital sites and differed significantly across primary medical or surgical diagnosis. The effect of primary medical or surgical diagnosis was larger than that of prior psychiatric history on Research Domain Criteria symptom burden.

CONCLUSION

Research Domain Criteria-based neuropsychiatric symptom burden estimated from general hospital patients' clinical documentation is more strongly associated with the primary hospital medical or surgical diagnosis than it is with the presence of a previous psychiatric history. The bidirectional role of psychiatric and somatic illness warrants further study through the lens of transdiagnostic phenotyping.

Cognitive measures lacking in EHR prior to dementia or Alzheimer's disease diagnosis

Introduction

The extent that cognitive measures are documented in electronic health records (EHR) is important for quality care and addressing disparities in timely diagnosis of dementia or Alzheimer's disease (AD).

Methods

Analysis of U.S. EHR data to describe the frequency and factors associated with cognitive measures prior to diagnosis of dementia (N = 111,125) or AD (N = 30,203).

Results

Only 11% of dementia patients and 24% of AD patients had a cognitive measure documented in the 5 years prior to diagnosis. Black race, older age, non‐commercial health insurance, lower mean neighborhood income, greater in‐patient stays, and fewer out‐patient visits were associated with lacking cognitive measures.

Discussion

Extensive missing cognitive data and differences in the availability of cognitive measures by race, age, and socioeconomic factors hinder patient care and limit utility of EHR for dementia research. Structured fields and prompts for cognitive data inputs at the point of care may help address these gaps.

Electronic Medical Record-Based Case Phenotyping for the Charlson Conditions: Scoping Review

BACKGROUND

Electronic medical records (EMRs) contain large amounts of rich clinical information. Developing EMR-based case definitions, also known as EMR phenotyping, is an active area of research that has implications for epidemiology, clinical care, and health services research.

OBJECTIVE

This review aims to describe and assess the present landscape of EMR-based case phenotyping for the Charlson conditions.

METHODS

A scoping review of EMR-based algorithms for defining the Charlson comorbidity index conditions was completed. This study covered articles published between January 2000 and April 2020, both inclusive. Embase (Excerpta Medica database) and MEDLINE (Medical Literature Analysis and Retrieval System Online) were searched using keywords developed in the following 3 domains: terms related to EMR, terms related to case finding, and disease-specific terms. The manuscript follows the Preferred Reporting Items for Systematic reviews and Meta-analyses extension for Scoping Reviews (PRISMA) guidelines.

RESULTS

A total of 274 articles representing 299 algorithms were assessed and summarized. Most studies were undertaken in the United States (181/299, 60.5%), followed by the United Kingdom (42/299, 14.0%) and Canada (15/299, 5.0%). These algorithms were mostly developed either in primary care (103/299, 34.4%) or inpatient (168/299, 56.2%) settings. Diabetes, congestive heart failure, myocardial infarction, and rheumatology had the highest number of developed algorithms. Data-driven and clinical rule-based approaches have been identified. EMR-based phenotype and algorithm development reflect the data access allowed by respective health systems, and algorithms vary in their performance.

CONCLUSIONS

Recognizing similarities and differences in health systems, data collection strategies, extraction, data release protocols, and existing clinical pathways is critical to algorithm development strategies. Several strategies to assist with phenotype-based case definitions have been proposed.

Indicators of dementia disease progression in primary care: An electronic health record cohort study

BACKGROUND AND PURPOSE

The objectives were to assess the feasibility and validity of using markers of dementia-related health as indicators of dementia progression in primary care, by assessing the frequency with which they are recorded and by testing the hypothesis that they are associated with recognised outcomes of dementia. The markers, in 13 domains, were derived previously through literature review, expert consensus, and analysis of regional primary care records.

METHODS

The study population consisted of patients with a recorded dementia diagnosis in the Clinical Practice Research Datalink, a UK primary care database linked to secondary care records. Incidence of recorded domains in the 36 months after diagnosis was determined. Associations of recording of domains with future hospital admission, palliative care, and mortality were derived.

RESULTS

There were 30,463 people with diagnosed dementia. Incidence of domains ranged from 469/1000 person-years (Increased Multimorbidity) to 11/1000 (Home Pressures). An increasing number of domains in which a new marker was recorded in the first year after diagnosis was associated with hospital admission (hazard ratio for ≥4 domains vs. no domains = 1.24; 95% confidence interval = 1.15-1.33), palliative care (1.87; 1.62-2.15), and mortality (1.57; 1.47-1.67). Individual domains were associated with outcomes with varying strengths of association.

CONCLUSIONS

Feasibility and validity of potential indicators of progression of dementia derived from primary care records are supported by their frequency of recording and associations with recognised outcomes. Further research should assess whether these markers can help identify patients with poorer prognosis to improve outcomes through stratified care and targeted support.

Distribution of agitation and related symptoms among hospitalized patients using a scalable natural language processing method

BACKGROUND

Agitation is a common feature of many neuropsychiatric disorders.

OBJECTIVE

Understanding the prevalence, implications, and characteristics of agitation among hospitalized populations can facilitate more precise recognition of disability arising from neuropsychiatric diseases.

METHODS

We developed two agitation phenotypes using an expansion of expert curated term lists. These phenotypes were used to characterize five years of psychiatric admissions. The relationship of agitation symptoms and length of stay was examined.

RESULTS

Among 4548 psychiatric admissions, 1134 (24.9%) included documentation of agitation based on the primary agitation phenotype. These symptoms were greater among individuals with public insurance, and those with mania and psychosis compared to major depressive disorder. Greater symptoms were associated with longer hospital stay, with ~0.9 day increase in stay for every 10% increase in agitation phenotype.

CONCLUSION

Agitation was common at hospital admission and associated with diagnosis and longer length of stay. Characterizing agitation-related symptoms through natural language processing may provide new tools for understanding agitated behaviors and their relationship to delirium.

Differences among Research Domain Criteria score trajectories by Diagnostic and Statistical Manual categorical diagnosis during inpatient hospitalization

With brief psychiatric hospitalizations, the extent to which symptoms change is rarely characterized. We sought to understand symptomatic changes across Research Domain Criteria (RDoC) dimensions, and the extent to which such improvement might be associated with risk for readmission. We identified 3,634 individuals with 4,713 hospital admissions to the psychiatric inpatient unit of a large academic medical center between 2010 and 2015. We applied a natural language processing tool to extract estimates of the five RDoC domains to the admission note and discharge summary and calculated the change in each domain. We examined the extent to which symptom domains changed during admission, and their relationship to baseline clinical and sociodemographic features, using linear regression. Symptomatic worsening was rare in the negative valence (0.4%) and positive valence (5.1%) domains, but more common in cognition (25.8%). Most diagnoses exhibited improvement in negative valence, which was associated with significant reduction in readmission risk. Despite generally brief hospital stays, we detected reduction across multiple symptom domains, with greatest improvement in negative symptoms, and greatest probability of worsening in cognitive symptoms. This approach should facilitate investigations of other features or interventions which may influence pace of clinical improvement.

Application of natural language processing algorithms for extracting information from news articles in event-based surveillance

The focus of this article is the application of natural language processing (NLP) for information extraction in event-based surveillance (EBS) systems. We describe common information extraction applications from open-source news articles and media sources in EBS systems, methods, value in public health, challenges and emerging developments.