Identifying risk of opioid use disorder for patients taking opioid medications with deep learning

Predictive Models to Assess Risk of Persistent Opioid Use, Opioid Use Disorder, and Overdose

BACKGROUND

This systematic review summarizes the development, accuracy, quality, and clinical utility of predictive models to assess the risk of opioid use disorder (OUD), persistent opioid use, and opioid overdose.

METHODS

In accordance with Preferred Reporting Items for a Systematic Review and Meta-analysis guidelines, 8 electronic databases were searched for studies on predictive models and OUD, overdose, or persistent use in adults until June 25, 2023. Study selection and data extraction were completed independently by 2 reviewers. Risk of bias of included studies was assessed independently by 2 reviewers using the Prediction model Risk of Bias ASsessment Tool (PROBAST).

RESULTS

The literature search yielded 3130 reports; after removing 199 duplicates, excluding 2685 studies after abstract review, and excluding 204 studies after full-text review, the final sample consisted of 41 studies that developed more than 160 predictive models. Primary outcomes included opioid overdose (31.6% of studies), OUD (41.4%), and persistent opioid use (17%). The most common modeling approach was regression modeling, and the most common predictors included age, sex, mental health diagnosis history, and substance use disorder history. Most studies reported model performance via the c statistic, ranging from 0.507 to 0.959; gradient boosting tree models and neural network models performed well in the context of their own study. One study deployed a model in real time. Risk of bias was predominantly high; concerns regarding applicability were predominantly low.

CONCLUSIONS

Models to predict opioid-related risks are developed using diverse data sources and predictors, with a wide and heterogenous range of accuracy metrics. There is a need for further research to improve their accuracy and implementation.

Predicting premature discontinuation of medication for opioid use disorder from electronic medical records

Medications such as buprenorphine-naloxone are among the most effective treatments for opioid use disorder, but limited retention in treatment limits long-term outcomes. In this study, we assess the feasibility of a machine learning model to predict retention vs. attrition in medication for opioid use disorder (MOUD) treatment using electronic medical record data including concepts extracted from clinical notes. A logistic regression classifier was trained on 374 MOUD treatments with 68% resulting in potential attrition. On a held-out test set of 157 events, the full model achieved an area under the receiver operating characteristic curve (AUROC) of 0.77 (95% CI: 0.64-0.90) and AUROC of 0.74 (95% CI: 0.62-0.87) with a limited model using only structured EMR data. Risk prediction for opioid MOUD retention vs. attrition is feasible given electronic medical record data, even without necessarily incorporating concepts extracted from clinical notes.

Assessment of Prediction Tasks and Time Window Selection in Temporal Modeling of Electronic Health Record Data: a Systematic Review

Temporal electronic health record (EHR) data are often preferred for clinical prediction tasks because they offer more complete representations of a patient's pathophysiology than static data. A challenge when working with temporal EHR data is problem formulation, which includes defining the time windows of interest and the prediction task. Our objective was to conduct a systematic review that assessed the definition and reporting of concepts relevant to temporal clinical prediction tasks. We searched PubMed® and IEEE Xplore® databases for studies from January 1, 2010 applying machine learning models to EHR data for patient outcome prediction. Publications applying time-series methods were selected for further review. We identified 92 studies and summarized them by clinical context and definition and reporting of the prediction problem. For the time windows of interest, 12 studies did not discuss window lengths, 57 used a single set of window lengths, and 23 evaluated the relationship between window length and model performance. We also found that 72 studies had appropriate reporting of the prediction task. However, evaluation of prediction problem formulation for temporal EHR data was complicated by heterogeneity in assessing and reporting of these concepts. Even among studies modeling similar clinical outcomes, there were variations in terminology used to describe the prediction problem, rationale for window lengths, and determination of the outcome of interest. As temporal modeling using EHR data expands, minimal reporting standards should include time-series specific concerns to promote rigor and reproducibility in future studies and facilitate model implementation in clinical settings.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00143-4.

Development of a neural network model to predict the presence of fentanyl in community drug samples

INTRODUCTION

Increasingly, Fourier-transform infrared (FTIR) spectroscopy is being used as a harm reduction tool to provide people who use drugs real-time information about the contents of their substances. However, FTIR spectroscopy has been shown to have a high detection limit for fentanyl and interpretation of results by a technician can be subjective. This poses concern, given that some synthetic opioids can produce serious toxicity at sub-detectable levels. The objective of this study was to develop a neural network model to identify fentanyl and related analogues more accurately in drug samples compared to traditional analysis by technicians.

METHODS

Data were drawn from samples analyzed point-of-care using combination FTIR spectroscopy and fentanyl immunoassay strips in British Columbia between August 2018 and January 2021. We developed neural network models to predict the presence of fentanyl based on FTIR data. The final model was validated against the results from immunoassay strips. Prediction performance was assessed using F1 score, accuracy, and area under the receiver-operating characteristic curve (AUROC), and was compared to results obtained from analysis by technicians.

RESULTS

A total of 12,684 samples were included. The neural network model outperformed results from those analyzed by technicians, with an F1 score of 96.4% and an accuracy of 96.4%, compared to 78.4% and 82.4% with a technician, respectively. The AUROC of the model was 99.0%. Fentanyl positive samples correctly detected by the model but not by the technician were typically those with low fentanyl concentrations (median: 2.3% quantity by weight; quartile 1-3: 0.0%-4.6%).

DISCUSSION

Neural network models can accurately predict the presence of fentanyl and related analogues using FTIR data, including samples with low fentanyl concentrations. Integrating this tool within drug checking services utilizing FTIR spectroscopy has the potential to improve decision making to reduce the risk of overdose and other negative health outcomes.

Machine learning for predicting opioid use disorder from healthcare data: A systematic review

INTRODUCTION

The US opioid epidemic has been one of the leading causes of injury-related deaths according to the CDC Injury Center. The increasing availability of data and tools for machine learning (ML) resulted in more researchers creating datasets and models to help analyze and mitigate the crisis. This review investigates peer-reviewed journal papers that applied ML models to predict opioid use disorder (OUD). The review is split into two parts. The first part summarizes the current research in OUD prediction with ML. The second part evaluates how ML techniques and processes were used to achieve these results and suggests improvements to refine further attempts to use ML for OUD prediction.

METHODS

The review includes peer-reviewed journal papers published on or after 2012 that use healthcare data to predict OUD. We searched Google Scholar, Semantic Scholar, PubMed, IEEE Xplore, and Science.gov in September of 2022. Data extracted includes the study's goal, dataset used, cohort selected, types of ML models created, model evaluation metrics, and the details of the ML tools and techniques used to create the models.

RESULTS

The review analyzed 16 papers. Three papers created their dataset, five used a publicly available dataset, and the remaining eight used a private dataset. Cohort size ranged from the low hundreds to over half a million. Six papers used one type of ML model, and the remaining ten used up to five different ML models. The reported ROC AUC was higher than 0.8 for all but one of the papers. Five papers used only non-interpretable models, and the other 11 used interpretable models exclusively or in combination with non-interpretable ones. The interpretable models were the highest or second-highest ROC AUC values. Most papers did not sufficiently describe the ML techniques and tools used to produce their results. Only three papers published their source code.

CONCLUSIONS

We found that while there are indications that ML methods applied to OUD prediction may be valuable, the lack of details and transparency in creating the ML models limits their usefulness. We end the review with recommendations to improve studies on this critical healthcare subject.

A hierarchical multilabel graph attention network method to predict the deterioration paths of chronic hepatitis B patients

OBJECTIVE

Estimating the deterioration paths of chronic hepatitis B (CHB) patients is critical for physicians' decisions and patient management. A novel, hierarchical multilabel graph attention-based method aims to predict patient deterioration paths more effectively. Applied to a CHB patient data set, it offers strong predictive utilities and clinical value.

MATERIALS AND METHODS

The proposed method incorporates patients' responses to medications, diagnosis event sequences, and outcome dependencies to estimate deterioration paths. From the electronic health records maintained by a major healthcare organization in Taiwan, we collect clinical data about 177 959 patients diagnosed with hepatitis B virus infection. We use this sample to evaluate the proposed method's predictive efficacy relative to 9 existing methods, as measured by precision, recall, F-measure, and area under the curve (AUC).

RESULTS

We use 20% of the sample as holdouts to test each method's prediction performance. The results indicate that our method consistently and significantly outperforms all benchmark methods. It attains the highest AUC, with a 4.8% improvement over the best-performing benchmark, as well as 20.9% and 11.4% improvements in precision and F-measures, respectively. The comparative results demonstrate that our method is more effective for predicting CHB patients' deterioration paths than existing predictive methods.

DISCUSSION AND CONCLUSION

The proposed method underscores the value of patient-medication interactions, temporal sequential patterns of distinct diagnosis, and patient outcome dependencies for capturing dynamics that underpin patient deterioration over time. Its efficacious estimates grant physicians a more holistic view of patient progressions and can enhance their clinical decision-making and patient management.

An integrated LSTM-HeteroRGNN model for interpretable opioid overdose risk prediction

Opioid overdose (OD) has become a leading cause of accidental death in the United States, and overdose deaths reached a record high during the COVID-19 pandemic. Combating the opioid crisis requires targeting high-need populations by identifying individuals at risk of OD. While deep learning emerges as a powerful method for building predictive models using large scale electronic health records (EHR), it is challenged by the complex intrinsic relationships among EHR data. Further, its utility is limited by the lack of clinically meaningful explainability, which is necessary for making informed clinical or policy decisions using such models. In this paper, we present LIGHTED, an integrated deep learning model combining long short term memory (LSTM) and graph neural networks (GNN) to predict patients' OD risk. The LIGHTED model can incorporate the temporal effects of disease progression and the knowledge learned from interactions among clinical features. We evaluated the model using Cerner's Health Facts database with over 5 million patients. Our experiments demonstrated that the model outperforms traditional machine learning methods and other deep learning models. We also proposed a novel interpretability method by exploiting embeddings provided by GNNs to cluster patients and EHR features respectively, and conducted qualitative feature cluster analysis for clinical interpretations. Our study shows that LIGHTED can take advantage of longitudinal EHR data and the intrinsic graph structure of EHRs among patients to provide effective and interpretable OD risk predictions that may potentially improve clinical decision support.

Narrative Review of Machine Learning in Rheumatic and Musculoskeletal Diseases for Clinicians and Researchers: Biases, Goals, and Future Directions

There has been rapid growth in the use of artificial intelligence (AI) analytics in medicine in recent years, including in rheumatic and musculoskeletal diseases (RMDs). Such methods represent a challenge to clinicians, patients, and researchers, given the "black box" nature of most algorithms, the unfamiliarity of the terms, and the lack of awareness of potential issues around these analyses. Therefore, this review aims to introduce this subject area in a way that is relevant and meaningful to clinicians and researchers. We hope to provide some insights into relevant strengths and limitations, reporting guidelines, as well as recent examples of such analyses in key areas, with a focus on lessons learned and future directions in diagnosis, phenotyping, prognosis, and precision medicine in RMDs.

Development and multimodal validation of a substance misuse algorithm for referral to treatment using artificial intelligence (SMART-AI): a retrospective deep learning study

Background

Substance misuse is a heterogeneous and complex set of behavioural conditions that are highly prevalent in hospital settings and frequently co-occur. Few hospital-wide solutions exist to comprehensively and reliably identify these conditions to prioritise care and guide treatment. The aim of this study was to apply natural language processing (NLP) to clinical notes collected in the electronic health record (EHR) to accurately screen for substance misuse.

Methods

The model was trained and developed on a reference dataset derived from a hospital-wide programme at Rush University Medical Center (RUMC), Chicago, IL, USA, that used structured diagnostic interviews to manually screen admitted patients over 27 months (between Oct 1, 2017, and Dec 31, 2019; n=54 915). The Alcohol Use Disorder Identification Test and Drug Abuse Screening Tool served as reference standards. The first 24 h of notes in the EHR were mapped to standardised medical vocabulary and fed into single-label, multilabel, and multilabel with auxillary-task neural network models. Temporal validation of the model was done using data from the subsequent 12 months on a subset of RUMC patients (n=16 917). External validation was done using data from Loyola University Medical Center, Chicago, IL, USA between Jan 1, 2007, and Sept 30, 2017 (n=1991 adult patients). The primary outcome was discrimination for alcohol misuse, opioid misuse, or non-opioid drug misuse. Discrimination was assessed by the area under the receiver operating characteristic curve (AUROC). Calibration slope and intercept were measured with the unreliability index. Bias assessments were performed across demographic subgroups.

Findings

The model was trained on a cohort that had 3·5% misuse (n=1 921) with any type of substance. 220 (11%) of 1921 patients with substance misuse had more than one type of misuse. The multilabel convolutional neural network classifier had a mean AUROC of 0·97 (95% CI 0·96–0·98) during temporal validation for all types of substance misuse. The model was well calibrated and showed good face validity with model features containing explicit mentions of aberrant drug-taking behaviour. A false-negative rate of 0·18–0·19 and a false-positive rate of 0·03 between non-Hispanic Black and non-Hispanic White groups occurred. In external validation, the AUROCs for alcohol and opioid misuse were 0·88 (95% CI 0·86–0·90) and 0·94 (0·92–0·95), respectively.

Interpretation

We developed a novel and accurate approach to leveraging the first 24 h of EHR notes for screening multiple types of substance misuse.

Funding

National Institute On Drug Abuse, National Institutes of Health.

A comprehensive mobile health intervention to prevent and manage the complexities of opioid use

OBJECTIVES

The Opioid Use crisis continues to be an epidemic with multiple known influencing and interacting factors. With the need for suitable opioid use interventions, we present a conceptual design of an m-health intervention that addresses the various known interacting factors of opioid use and corresponding evidence-based practices. The visualization of the opioid use complexities is presented as the "Opioid Cube".

METHODS

Following Stage 0 to Stage IA of the NIH Stage Model, we used guidelines and extant health intervention literature on opioid apps to inform the Opioid Intervention (O-INT) design. We present our design using system architecture, algorithms, and user interfaces to integrate multiple functions including decision support. We evaluate the proposed O-INT using analytical modeling.

RESULTS

The conceptual design of O-INT supports the concept of collaborative care, by providing connections between the patient, healthcare professionals, and their family members. The evaluation of O-INT shows a preference for specific functions, such as overdose detection and potential for high system reliability with minimal side effects. The Opioid Cube provides a visualization of various opioid use states and their influencing and interacting factors.

CONCLUSIONS

O-INT is a promising design with a holistic approach to manage opioid use and prevent and treat misuse. With several needed functionalities, O-INT design serves as a decision support system for patients, healthcare professionals, researchers, and policy makers. Together, O-INT and the Opioid Cube may serve as a foundation for development and adoption of highly effective m-health interventions for opioid use.

Towards the Development of a Substance Abuse Index (SEI) through Informatics

Substance abuse or drug dependence is a prevalent phenomenon, and is on the rise in United States. Important contributing factors for the prevalence are the addictive nature of certain medicinal/prescriptive drugs, individual dispositions (biological, physiological, and psychological), and other external influences (e.g., pharmaceutical advertising campaigns). However, currently there is no comprehensive computational or machine learning framework that allows systematic studies of substance abuse and its factors with majority of the works using subjective surveys questionnaires and focusing on classification techniques. Lacking standardized methods and/or measures to prescribe medication and to study substance abuse makes it difficult to advance through collective research efforts. Thus, in this paper, we propose to test the feasibility of developing a, objective substance effect index, SEI, that can measure the tendency of an individual towards substance abuse. To that end, we combine the benefits of Electronics Medical Records (EMR) with machine learning technology by defining SEI as a function of EMR data and using logistics regression to obtain a closed form expression for SEI. We conduct various evaluations to validate the proposed model, and the results show that further work towards the development of SEI will not only provide researchers with standard computational measure for substance abuse, but may also allow them to study certain attribute interactions to gain further insights into substance abuse tendencies.