Development of a machine learning algorithm for early detection of opioid use disorder

Predictive Models to Assess Risk of Persistent Opioid Use, Opioid Use Disorder, and Overdose

BACKGROUND

This systematic review summarizes the development, accuracy, quality, and clinical utility of predictive models to assess the risk of opioid use disorder (OUD), persistent opioid use, and opioid overdose.

METHODS

In accordance with Preferred Reporting Items for a Systematic Review and Meta-analysis guidelines, 8 electronic databases were searched for studies on predictive models and OUD, overdose, or persistent use in adults until June 25, 2023. Study selection and data extraction were completed independently by 2 reviewers. Risk of bias of included studies was assessed independently by 2 reviewers using the Prediction model Risk of Bias ASsessment Tool (PROBAST).

RESULTS

The literature search yielded 3130 reports; after removing 199 duplicates, excluding 2685 studies after abstract review, and excluding 204 studies after full-text review, the final sample consisted of 41 studies that developed more than 160 predictive models. Primary outcomes included opioid overdose (31.6% of studies), OUD (41.4%), and persistent opioid use (17%). The most common modeling approach was regression modeling, and the most common predictors included age, sex, mental health diagnosis history, and substance use disorder history. Most studies reported model performance via the c statistic, ranging from 0.507 to 0.959; gradient boosting tree models and neural network models performed well in the context of their own study. One study deployed a model in real time. Risk of bias was predominantly high; concerns regarding applicability were predominantly low.

CONCLUSIONS

Models to predict opioid-related risks are developed using diverse data sources and predictors, with a wide and heterogenous range of accuracy metrics. There is a need for further research to improve their accuracy and implementation.

Machine Learned Classification of Ligand Intrinsic Activity at Human μ-Opioid Receptor

Opioids are small-molecule agonists of μ-opioid receptor (μOR), while reversal agents such as naloxone are antagonists of mOR. Here we developed machine learning models to classify the intrinsic activities of ligands at the human μOR. We first manually curated a database of 983 small molecules with measured E m a x values at the human μOR. Analysis of the chemical space allowed identification of dominant scaffolds and structurally similar agonists and antagonists. Decision tree models and directed message passing neural networks (MPNNs) were then trained to classify agonistic and antagonistic ligands. The hold-out test AUCs of the extra-tree (ET) and MPNN models are 91.5±3.9% and 91.8±4.4%, respectively, while the respective balanced accuracies (BAs) are 83.3±5.0% and 85.1±5.0%. To overcome the challenge of small dataset, a student-teacher learning method called tri-training with disagreement was tested using an unlabeled dataset comprised of 15,816 ligands of human, mouse, or rat μOR, κOR, or δOR. We found that the tri-training scheme was able to increase the MPNN AUC to as high as 9.7%. Taken together, our work provides a proof of concept for developing machine learning models to predict μOR ligand intrinsic activities despite small data size. We envisage many future applications of these models, including evaluation of pharmacologically uncharacterized substances that may pose a risk to public safety and discovery of new rescue agents to combat opioid overdoses.

Identifying patients with opioid use disorder using International Classification of Diseases (ICD) codes: Challenges and opportunities

BACKGROUND AND AIMS

International Classification of Diseases (ICD) diagnosis codes are often used in research to identify patients with opioid use disorder (OUD), but their accuracy for this purpose is not fully evaluated. This study describes application of ICD-10 diagnosis codes for opioid use, dependence and abuse from an electronic health record (EHR) data extraction using data from the clinics' OUD patient registries and clinician/staff EHR entries.

DESIGN

Cross-sectional observational study.

SETTING

Four rural primary care clinics in Washington and Idaho, USA.

PARTICIPANTS

307 patients.

MEASUREMENTS

This study used three data sources from each clinic: (1) a limited dataset extracted from the EHR, (2) a clinic-based registry of patients with OUD and (3) the clinician/staff interface of the EHR (e.g. progress notes, problem list). Data source one included records with six commonly applied ICD-10 codes for opioid use, dependence and abuse: F11.10 (opioid abuse, uncomplicated), F11.20 (opioid dependence, uncomplicated), F11.21 (opioid dependence, in remission), F11.23 (opioid dependence with withdrawal), F11.90 (opioid use, unspecified, uncomplicated) and F11.99 (opioid use, unspecified with unspecified opioid-induced disorder). Care coordinators used data sources two and three to categorize each patient identified in data source one: (1) confirmed OUD diagnosis, (2) may have OUD but no confirmed OUD diagnosis, (3) chronic pain with no evidence of OUD and (4) no evidence for OUD or chronic pain.

FINDINGS

F11.10, F11.21 and F11.99 were applied most frequently to patients who had clinical diagnoses of OUD (64%, 89% and 79%, respectively). F11.20, F11.23 and F11.90 were applied to patients who had a diagnostic mix of OUD and chronic pain without OUD. The four clinics applied codes inconsistently.

CONCLUSIONS

Lack of uniform application of ICD diagnosis codes make it challenging to use diagnosis code data from EHR to identify a research population of persons with opioid use disorder.

The evolution of Big Data in neuroscience and neurology

Neurological diseases are on the rise worldwide, leading to increased healthcare costs and diminished quality of life in patients. In recent years, Big Data has started to transform the fields of Neuroscience and Neurology. Scientists and clinicians are collaborating in global alliances, combining diverse datasets on a massive scale, and solving complex computational problems that demand the utilization of increasingly powerful computational resources. This Big Data revolution is opening new avenues for developing innovative treatments for neurological diseases. Our paper surveys Big Data's impact on neurological patient care, as exemplified through work done in a comprehensive selection of areas, including Connectomics, Alzheimer's Disease, Stroke, Depression, Parkinson's Disease, Pain, and Addiction (e.g., Opioid Use Disorder). We present an overview of research and the methodologies utilizing Big Data in each area, as well as their current limitations and technical challenges. Despite the potential benefits, the full potential of Big Data in these fields currently remains unrealized. We close with recommendations for future research aimed at optimizing the use of Big Data in Neuroscience and Neurology for improved patient outcomes.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40537-023-00751-2.

Opioid-related harms and care impacts of conventional and AI-based prescription management strategies: insights from leveraging agent-based modeling and machine learning

Introduction

Like its counterpart to the south, Canada ranks among the top five countries with the highest rates of opioid prescriptions. With many suffering from opioid use disorder first having encountered opioids via prescription routes, practitioners and health systems have an enduring need to identify and effectively respond to the problematic use of opioid prescription. There are strong challenges to successfully addressing this need: importantly, the patterns of prescription fulfillment that signal opioid abuse can be subtle and difficult to recognize, and overzealous enforcement can deprive those with legitimate pain management needs the appropriate care. Moreover, injudicious responses risk shifting those suffering from early-stage abuse of prescribed opioids to illicitly sourced street alternatives, whose varying dosage, availability, and the risk of adulteration can pose grave health risks.

Methods

This study employs a dynamic modeling and simulation to evaluate the effectiveness of prescription regimes employing machine learning monitoring programs to identify the patients who are at risk of opioid abuse while being treated with prescribed opioids. To this end, an agent-based model was developed and implemented to examine the effect of reduced prescribing and prescription drug monitoring programs on overdose and escalation to street opioids among patients, and on the legitimacy of fulfillments of opioid prescriptions over a 5-year time horizon. A study released by the Canadian Institute for Health Information was used to estimate the parameter values and assist in the validation of the existing agent-based model.

Results and discussion

The model estimates that lowering the prescription doses exerted the most favorable impact on the outcomes of interest over 5 years with a minimum burden on patients with a legitimate need for pharmaceutical opioids. The accurate conclusion about the impact of public health interventions requires a comprehensive set of outcomes to test their multi-dimensional effects, as utilized in this research. Finally, combining machine learning and agent-based modeling can provide significant advantages, particularly when using the latter to gain insights into the long-term effects and dynamic circumstances of the former.

Machine learning for predicting opioid use disorder from healthcare data: A systematic review

INTRODUCTION

The US opioid epidemic has been one of the leading causes of injury-related deaths according to the CDC Injury Center. The increasing availability of data and tools for machine learning (ML) resulted in more researchers creating datasets and models to help analyze and mitigate the crisis. This review investigates peer-reviewed journal papers that applied ML models to predict opioid use disorder (OUD). The review is split into two parts. The first part summarizes the current research in OUD prediction with ML. The second part evaluates how ML techniques and processes were used to achieve these results and suggests improvements to refine further attempts to use ML for OUD prediction.

METHODS

The review includes peer-reviewed journal papers published on or after 2012 that use healthcare data to predict OUD. We searched Google Scholar, Semantic Scholar, PubMed, IEEE Xplore, and Science.gov in September of 2022. Data extracted includes the study's goal, dataset used, cohort selected, types of ML models created, model evaluation metrics, and the details of the ML tools and techniques used to create the models.

RESULTS

The review analyzed 16 papers. Three papers created their dataset, five used a publicly available dataset, and the remaining eight used a private dataset. Cohort size ranged from the low hundreds to over half a million. Six papers used one type of ML model, and the remaining ten used up to five different ML models. The reported ROC AUC was higher than 0.8 for all but one of the papers. Five papers used only non-interpretable models, and the other 11 used interpretable models exclusively or in combination with non-interpretable ones. The interpretable models were the highest or second-highest ROC AUC values. Most papers did not sufficiently describe the ML techniques and tools used to produce their results. Only three papers published their source code.

CONCLUSIONS

We found that while there are indications that ML methods applied to OUD prediction may be valuable, the lack of details and transparency in creating the ML models limits their usefulness. We end the review with recommendations to improve studies on this critical healthcare subject.

Individualized Prospective Prediction of Opioid Use Disorder

OBJECTIVE

Opioid use disorder (OUD) is a chronic relapsing disorder with a problematic pattern of opioid use, affecting nearly 27 million people worldwide. Machine learning (ML)-based prediction of OUD may lead to early detection and intervention. However, most ML prediction studies were not based on representative data sources and prospective validations, limiting their potential to predict future new cases. In the current study, we aimed to develop and prospectively validate an ML model that could predict individual OUD cases based on representative large-scale health data.

METHOD

We present an ensemble machine-learning model trained on a cross-linked Canadian administrative health data set from 2014 to 2018 (n  =  699,164), with validation of model-predicted OUD cases on a hold-out sample from 2014 to 2018 (n  =  174,791) and prospective prediction of OUD cases on a non-overlapping sample from 2019 (n  =  316,039). We used administrative records of OUD diagnosis for each subject based on International Classification of Diseases (ICD) codes.

RESULTS

With 6409 OUD cases in 2019 (mean [SD], 45.34 [14.28], 3400 males), our model prospectively predicted OUD cases at a high accuracy (balanced accuracy, 86%, sensitivity, 93%; specificity 79%). In accord with prior findings, the top risk factors for OUD in this model were opioid use indicators and a history of other substance use disorders.

CONCLUSION

Our study presents an individualized prospective prediction of OUD cases by applying ML to large administrative health datasets. Such prospective predictions based on ML would be essential for potential future clinical applications in the early detection of OUD.

Machine Learning Model Identifies Preoperative Opioid Use, Male Sex, and Elevated BMI as Predictive Factors for of Prolonged Opioid Consumption Following Arthroscopic Meniscal Surgery

PURPOSE

To develop a predictive machine learning model to identify prognostic factors for continued opioid prescriptions after arthroscopic meniscus surgery.

METHODS

Patients undergoing arthroscopic meniscal surgery, such as meniscus debridement, repair, or revision at a single institution from 2013 to 2017 were retrospectively followed up to 1 year postoperatively. Procedural details were recorded, including concomitant procedures, primary versus revision, and whether a partial debridement or a repair was performed. Intraoperative arthritis severity was measured using the Outerbridge Classification. The number of opioid prescriptions in each month was recorded. Primary analysis used was the multivariate Cox-Regression model. We then created a naïve Bayesian model, a machine learning classifier that uses Bayes' theorem with an assumption of independence between variables.

RESULTS

A total of 581 patients were reviewed. Postoperative opioid refills occurred in 98 patients (16.9%). Multivariate logistic modeling was used; independent risk factors for opioid refills included male sex, larger body mass index, and chronic preoperative opioid use, while meniscus resection demonstrated decreased likelihood of refills. Concomitant procedures, revision procedures, and presence of arthritis graded by the Outerbridge classification were not significant predictors of postoperative opioid refills. The naïve Bayesian model for extended postoperative opioid use demonstrated good fit with our cohort with an area under the curve of 0.79, sensitivity of 94.5%, positive predictive value (PPV) of 83%, and a detection rate of 78.2%. The two most important features in the model were preoperative opioid use and male sex.

CONCLUSION

After arthroscopic meniscus surgery, preoperative opioid consumption and male sex were the most significant predictors for sustained opioid use beyond 1 month postoperatively. Intraoperative arthritis was not an independent risk factor for continued refills. A machine learning algorithm performed with high accuracy, although with a high false positive rate, to function as a screening tool to identify patients filling additional narcotic prescriptions after surgery.

LEVEL OF EVIDENCE

III, retrospective comparative study.

Artificial Intelligence in Emergency Medicine: Benefits, Risks, and Recommendations

BACKGROUND

Artificial intelligence (AI) can be described as the use of computers to perform tasks that formerly required human cognition. The American Medical Association prefers the term 'augmented intelligence' over 'artificial intelligence' to emphasize the assistive role of computers in enhancing physician skills as opposed to replacing them. The integration of AI into emergency medicine, and clinical practice at large, has increased in recent years, and that trend is likely to continue.

DISCUSSION

AI has demonstrated substantial potential benefit for physicians and patients. These benefits are transforming the therapeutic relationship from the traditional physician-patient dyad into a triadic doctor-patient-machine relationship. New AI technologies, however, require careful vetting, legal standards, patient safeguards, and provider education. Emergency physicians (EPs) should recognize the limits and risks of AI as well as its potential benefits.

CONCLUSIONS

EPs must learn to partner with, not capitulate to, AI. AI has proven to be superior to, or on a par with, certain physician skills, such as interpreting radiographs and making diagnoses based on visual cues, such as skin cancer. AI can provide cognitive assistance, but EPs must interpret AI results within the clinical context of individual patients. They must also advocate for patient confidentiality, professional liability coverage, and the essential role of specialty-trained EPs.

Identifying Opioid Use Disorder from Longitudinal Healthcare Data using a Multi-stream Transformer

Opioid Use Disorder (OUD) is a public health crisis costing the US billions of dollars annually in healthcare, lost workplace productivity, and crime. Analyzing longitudinal healthcare data is critical in addressing many real-world problems in healthcare. Leveraging the real-world longitudinal healthcare data, we propose a novel multi-stream transformer model called MUPOD for OUD identification. MUPOD is designed to simultaneously analyze multiple types of healthcare data streams, such as medications and diagnoses, by attending to segments within and across these data streams. Our model tested on the data from 392,492 patients with long-term back pain problems showed significantly better performance than the traditional models and recently developed deep learning models.