Identification of opioid use disorder using electronic health records: Beyond diagnostic codes

Automating the Addiction Behaviors Checklist for Problematic Opioid Use Identification

Importance

Individuals whose chronic pain is managed with opioids are at high risk of developing an opioid use disorder. Electronic health records (EHR) allow large-scale studies to identify a continuum of problematic opioid use, including opioid use disorder. Traditionally, this is done through diagnostic codes, which are often unreliable and underused.

Objective

To determine whether regular expressions, an interpretable natural language processing technique, could automate a validated clinical tool (Addiction Behaviors Checklist) to identify problematic opioid use.

Design, Setting, and Participants

This cross-sectional study reports on a retrospective cohort with data analyzed from 2021 through 2023. The approach was evaluated against a blinded, manually reviewed holdout test set and validated against an independent test set at a separate institution. The study used data from Vanderbilt University Medical Center's Synthetic Derivative, a deidentified version of the EHR for research purposes. This cohort comprised 8063 individuals with chronic pain, defined by diagnostic codes on at least 2 days. The study team collected free-text notes, demographics, and diagnostic codes and performed an external validation with 100 individuals with chronic pain from Geisinger, recruited from an interventional pain clinic cohort.

Main Outcomes and Measures

The primary outcome was the evaluation of the automated method in identifying patients demonstrating problematic opioid use and its comparison with manual medical record review and opioid use disorder diagnostic codes. Methods with F1 scores were evaluated (a single value that combines sensitivity and positive predictive value at a single threshold) and areas under the curve (a single value that combines sensitivity and specificity across multiple thresholds).

Results

Among the 8063 patients in the primary site (5081 female [63%] and 2982 male [37%]; mean [SD] age, 56 [16] years) and 100 patients in the validation site (57 female [57%] and 43 male [43%]; mean [SD] age, 54 [13] years), the automated approach outperformed diagnostic codes based on F1 scores (0.73; 95% CI, 0.62-0.83 vs 0.08; 95% CI, 0.00-0.19 at the primary site and 0.70; 95% CI, 0.50-0.85 vs 0.29; 95% CI, 0.07-0.50 at the validation site) and areas under the curve (0.82; 95% CI, 0.73-0.89 vs 0.52; 95% CI, 0.50-0.55 at the primary site and 0.86; 95% CI, 0.76-0.94 vs 0.59;95% CI, 0.50-0.67 at validation site).

Conclusions

This automated data extraction technique may facilitate earlier identification of people at risk for and who are experiencing problematic opioid use, and create new opportunities for studying long-term sequelae of opioid pain management.

Improving diagnosis-based quality measures: an application of machine learning to the prediction of substance use disorder among outpatients

OBJECTIVE

Substance use disorder (SUD) is clinically under-detected and under-documented. We built and validated machine learning (ML) models to estimate SUD prevalence from electronic health record (EHR) data and to assess variation in facility-level SUD identification using clinically documented diagnoses vs model-based estimated prevalence.

METHODS

Predictors included demographics, SUD-related diagnoses and healthcare utilisation. The criterion outcome for model development was prevalent SUD assessed via a patient survey across 30 geographically representative Veterans Health Administration (VA) sites (n=5989 patients). We split the data into training and testing datasets and built a series of ML models using cross-validation to minimise over-fitting. We selected the final model based on its performance in predicting SUD in the testing dataset. Using the final model, we estimated SUD prevalence at all 30 sites. We then compared facilities based on SUD identification using two alternative SUD identification measures: the facility-level SUD diagnosis rate and model-based estimated SUD prevalence.

RESULTS

The best-performing LASSO model with n=61 predictors doubled the sensitivity for classifying SUD relative to a model with only documented SUD diagnoses (0.682 vs 0.331). Across the 30 sites, SUD diagnosis rates ranged from 6.4%-13.9% and predicted SUD prevalence ranged from 9.7-16.0%. The difference in facility-level SUD identification (observed diagnosis rate minus predicted prevalence) ranged from -7.2 to +1.3 percentage points. Comparing facilities' rank ordering on documented SUD diagnosis rates vs estimated SUD prevalence, 16 out of 30 sites had a ranking that changed by at least a quintile (ie, 6 places or more).

CONCLUSIONS

This analysis shows that use of model-based performance measures may help address measurement blind spots that arise due to differences in diagnostic accuracy across sites. Although model-based estimates better estimate SUD prevalence relative to diagnoses alone for facility quality assessment, further improvements and individual SUD detection both require enhanced direct screening for non-alcohol drug use.

A Review of Leveraging Artificial Intelligence to Predict Persistent Postoperative Opioid Use and Opioid Use Disorder and its Ethical Considerations

PURPOSE OF REVIEW

Artificial intelligence (AI) offers a new frontier for aiding in the management of both acute and chronic pain, which may potentially transform opioid prescribing practices and addiction prevention strategies. In this review paper, not only do we discuss some of the current literature around predicting various opioid-related outcomes, but we also briefly point out the next steps to improve trustworthiness of these AI models prior to real-time use in clinical workflow.

RECENT FINDINGS

Machine learning-based predictive models for identifying risk for persistent postoperative opioid use have been reported for spine surgery, knee arthroplasty, hip arthroplasty, arthroscopic joint surgery, outpatient surgery, and mixed surgical populations. Several machine learning-based models have been described to predict an individual's propensity for opioid use disorder and opioid overdose. Natural language processing and large language model approaches have been described to detect opioid use disorder and persistent postsurgical opioid use from clinical notes. AI holds significant promise in enhancing the management of acute and chronic opioids, which may offer tools to help optimize dosing, predict addiction risks, and personalize pain management strategies. By harnessing the power of AI, healthcare providers can potentially improve patient outcomes, reduce the burden of opioid addiction, and contribute to solving the opioid crisis.

Developing a Framework to Infer Opioid Use Disorder Severity From Clinical Notes to Inform Natural Language Processing Methods: Characterization Study

BACKGROUND

Information regarding opioid use disorder (OUD) status and severity is important for patient care. Clinical notes provide valuable information for detecting and characterizing problematic opioid use, necessitating development of natural language processing (NLP) tools, which in turn requires reliably labeled OUD-relevant text and understanding of documentation patterns.

OBJECTIVE

To inform automated NLP methods, we aimed to develop and evaluate an annotation schema for characterizing OUD and its severity, and to document patterns of OUD-relevant information within clinical notes of heterogeneous patient cohorts.

METHODS

We developed an annotation schema to characterize OUD severity based on criteria from the Diagnostic and Statistical Manual of Mental Disorders, 5th edition. In total, 2 annotators reviewed clinical notes from key encounters of 100 adult patients with varied evidence of OUD, including patients with and those without chronic pain, with and without medication treatment for OUD, and a control group. We completed annotations at the sentence level. We calculated severity scores based on annotation of note text with 18 classes aligned with criteria for OUD severity and determined positive predictive values for OUD severity.

RESULTS

The annotation schema contained 27 classes. We annotated 1436 sentences from 82 patients; notes of 18 patients (11 of whom were controls) contained no relevant information. Interannotator agreement was above 70% for 11 of 15 batches of reviewed notes. Severity scores for control group patients were all 0. Among noncontrol patients, the mean severity score was 5.1 (SD 3.2), indicating moderate OUD, and the positive predictive value for detecting moderate or severe OUD was 0.71. Progress notes and notes from emergency department and outpatient settings contained the most and greatest diversity of information. Substance misuse and psychiatric classes were most prevalent and highly correlated across note types with high co-occurrence across patients.

CONCLUSIONS

Implementation of the annotation schema demonstrated strong potential for inferring OUD severity based on key information in a small set of clinical notes and highlighting where such information is documented. These advancements will facilitate NLP tool development to improve OUD prevention, diagnosis, and treatment.