Using Administrative Data to Predict Suicide After Psychiatric Hospitalization in the Veterans Health Administration System

Sexual and Gender Minority Status and Suicide Mortality: An Explainable Artificial Intelligence Analysis

Objectives: Suicide risk is elevated in lesbian, gay, bisexual, and transgender (LGBT) individuals. Limited data on LGBT status in healthcare systems hinder our understanding of this risk. This study used natural language processing to extract LGBT status and a deep neural network (DNN) to examine suicidal death risk factors among US Veterans. Methods: Data on 8.8 million veterans with visits between 2010 and 2017 was used. A case-control study was performed, and suicide death risk was analyzed by a DNN. Feature impacts and interactions on the outcome were evaluated. Results: The crude suicide mortality rate was higher in LGBT patients. However, after adjusting for over 200 risk and protective factors, known LGBT status was associated with reduced risk compared to LGBT-Unknown status. Among LGBT patients, black, female, married, and older Veterans have a higher risk, while Veterans of various religions have a lower risk. Conclusion: Our results suggest that disclosed LGBT status is not directly associated with an increase suicide death risk, however, other factors (e.g., depression and anxiety caused by stigma) are associated with suicide death risks.

Predicting Suicide Among US Veterans Using Natural Language Processing-enriched Social and Behavioral Determinants of Health

Despite recognizing the critical association between social and behavioral determinants of health (SBDH) and suicide risk, SBDHs from unstructured electronic health record (EHR) notes for suicide predictive modeling remain underutilized. This study investigates the impact of SBDH, identified from both structured and unstructured data utilizing a natural language processing (NLP) system, on suicide prediction within 7, 30, 90, and 180 days of discharge. Using EHR data of 2,987,006 Veterans between October 1, 2009, and September 30, 2015, from the US Veterans Health Administration (VHA), we designed a case-control study that demonstrates that incorporating structured and NLP-extracted SBDH significantly enhances the performance of three architecturally distinct suicide predictive models - elastic-net logistic regression, random forest (RF), and multilayer perceptron. For example, RF achieved notable improvements in suicide prediction within 180 days of discharge, with an increase in the area under the receiver operating characteristic curve from 83.57-84.25% (95% CI = 0.63%-0.98%, p-val < 0.001) and the area under the precision recall curve from 57.38-59.87% (95% CI = 3.86%-4.82%, p-val < 0.001) after integrating NLP-extracted SBDH. These findings underscore the potential of NLP-extracted SBDH in enhancing suicide prediction across various prediction timeframes, offering valuable insights for healthcare practitioners and policymakers.

Machine learning and the prediction of suicide in psychiatric populations: a systematic review

Machine learning (ML) has emerged as a promising tool to enhance suicidal prediction. However, as many large-sample studies mixed psychiatric and non-psychiatric populations, a formal psychiatric diagnosis emerged as a strong predictor of suicidal risk, overshadowing more subtle risk factors specific to distinct populations. To overcome this limitation, we conducted a systematic review of ML studies evaluating suicidal behaviors exclusively in psychiatric clinical populations. A systematic literature search was performed from inception through November 17, 2022 on PubMed, EMBASE, and Scopus following the PRISMA guidelines. Original research using ML techniques to assess the risk of suicide or predict suicide attempts in the psychiatric population were included. An assessment for bias risk was performed using the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines. About 1032 studies were retrieved, and 81 satisfied the inclusion criteria and were included for qualitative synthesis. Clinical and demographic features were the most frequently employed and random forest, support vector machine, and convolutional neural network performed better in terms of accuracy than other algorithms when directly compared. Despite heterogeneity in procedures, most studies reported an accuracy of 70% or greater based on features such as previous attempts, severity of the disorder, and pharmacological treatments. Although the evidence reported is promising, ML algorithms for suicidal prediction still present limitations, including the lack of neurobiological and imaging data and the lack of external validation samples. Overcoming these issues may lead to the development of models to adopt in clinical practice. Further research is warranted to boost a field that holds the potential to critically impact suicide mortality.

Deep sequential neural network models improve stratification of suicide attempt risk among US veterans

OBJECTIVE

To apply deep neural networks (DNNs) to longitudinal EHR data in order to predict suicide attempt risk among veterans. Local explainability techniques were used to provide explanations for each prediction with the goal of ultimately improving outreach and intervention efforts.

MATERIALS AND METHODS

The DNNs fused demographic information with diagnostic, prescription, and procedure codes. Models were trained and tested on EHR data of approximately 500 000 US veterans: all veterans with recorded suicide attempts from April 1, 2005, through January 1, 2016, each paired with 5 veterans of the same age who did not attempt suicide. Shapley Additive Explanation (SHAP) values were calculated to provide explanations of DNN predictions.

RESULTS

The DNNs outperformed logistic and linear regression models in predicting suicide attempts. After adjusting for the sampling technique, the convolutional neural network (CNN) model achieved a positive predictive value (PPV) of 0.54 for suicide attempts within 12 months by veterans in the top 0.1% risk tier. Explainability methods identified meaningful subgroups of high-risk veterans as well as key determinants of suicide attempt risk at both the group and individual level.

DISCUSSION AND CONCLUSION

The deep learning methods employed in the present study have the potential to significantly enhance existing suicide risk models for veterans. These methods can also provide important clues to explore the relative value of long-term and short-term intervention strategies. Furthermore, the explainability methods utilized here could also be used to communicate to clinicians the key features which increase specific veterans' risk for attempting suicide.

Identifying features of risk periods for suicide attempts using document frequency and language use in electronic health records

Background

Individualising mental healthcare at times when a patient is most at risk of suicide involves shifting research emphasis from static risk factors to those that may be modifiable with interventions. Currently, risk assessment is based on a range of extensively reported stable risk factors, but critical to dynamic suicide risk assessment is an understanding of each individual patient's health trajectory over time. The use of electronic health records (EHRs) and analysis using machine learning has the potential to accelerate progress in developing early warning indicators.

Setting

EHR data from the South London and Maudsley NHS Foundation Trust (SLaM) which provides secondary mental healthcare for 1.8 million people living in four South London boroughs.

Objectives

To determine whether the time window proximal to a hospitalised suicide attempt can be discriminated from a distal period of lower risk by analysing the documentation and mental health clinical free text data from EHRs and (i) investigate whether the rate at which EHR documents are recorded per patient is associated with a suicide attempt; (ii) compare document-level word usage between documents proximal and distal to a suicide attempt; and (iii) compare n-gram frequency related to third-person pronoun use proximal and distal to a suicide attempt using machine learning.

Methods

The Clinical Record Interactive Search (CRIS) system allowed access to de-identified information from the EHRs. CRIS has been linked with Hospital Episode Statistics (HES) data for Admitted Patient Care. We analysed document and event data for patients who had at some point between 1 April 2006 and 31 March 2013 been hospitalised with a HES ICD-10 code related to attempted suicide (X60-X84; Y10-Y34; Y87.0/Y87.2).

Findings

n = 8,247 patients were identified to have made a hospitalised suicide attempt. Of these, n = 3,167 (39.8%) of patients had at least one document available in their EHR prior to their first suicide attempt. N = 1,424 (45.0%) of these patients had been "monitored" by mental healthcare services in the past 30 days. From 60 days prior to a first suicide attempt, there was a rapid increase in the monitoring level (document recording of the past 30 days) increasing from 35.1 to 45.0%. Documents containing words related to prescribed medications/drugs/overdose/poisoning/addiction had the highest odds of being a risk indicator used proximal to a suicide attempt (OR 1.88; precision 0.91 and recall 0.93), and documents with words citing a care plan were associated with the lowest risk for a suicide attempt (OR 0.22; precision 1.00 and recall 1.00). Function words, word sequence, and pronouns were most common in all three representations (uni-, bi-, and tri-gram).

Conclusion

EHR documentation frequency and language use can be used to distinguish periods distal from and proximal to a suicide attempt. However, in our study 55.0% of patients with documentation, prior to their first suicide attempt, did not have a record in the preceding 30 days, meaning that there are a high number who are not seen by services at their most vulnerable point.

Prediction models of suicide and non-fatal suicide attempt after discharge from a psychiatric inpatient stay: A machine learning approach on nationwide Danish registers

INTRODUCTION

To develop machine learning models capable of predicting suicide and non-fatal suicide attempt as separate outcomes in the first 30 days after discharge from a psychiatric inpatient stay.

METHODS

Prospective cohort study using nationwide Danish registry data. We included individuals who were 18 years or older, and all discharges from psychiatric hospitalizations in Denmark from 1995 to 2018. We trained predictive models using 10-fold cross validation on 80% of the data and did testing on the remaining 20%.

RESULTS

The best model for predicting non-fatal suicide attempt was an ensemble of predictions from gradient boosting (XGBoost) and categorical boosting (catBoost). The ROC-AUC for predicting suicide attempt was 0.85 (95% CI: 0.84-0.85). At a risk threshold of 4.36%, positive predictive value (PPV) was 11.0% and sensitivity was 47.2%. The best model for predicting suicide was an ensemble of predictions from random forest, XGBoost and catBoost. For suicide, the ROC-AUC was 0.71 (95% CI: 0.70-0.73). At a risk threshold of 0.15%, PPV was 0.34% and sensitivity was 56.0%. The most contributing predictors differed when predicting suicide and suicide attempt, indicating that separate models are needed. The ensemble model was fair across sex and age, and more so than the penalized logistic regression model.

CONCLUSIONS

We achieved good performance for predicting suicide attempts and demonstrated a clinical application of ensemble models. Our results indicate a difference in predictive performance for models predicting suicide and suicide attempt, respectively. Thus, we recommend that suicide and suicide attempt are treated as two separate endpoints, in particular for clinical application. We demonstrated that the ensemble model is fairer across sex and age compared with a penalized logistic regression, and therefore we recommend the use of well-tested ensembles despite a more complex explainability.

Relationship Between Pain and LGBT Status Among Veterans in Care in a Retrospective Cross-Sectional Cohort

Background

Pain assessment is performed in many healthcare systems, such as the Veterans Health Administration, but prior studies have not assessed whether pain screening varies in sexual and gender minority populations that include individuals who identify as lesbian, gay, bisexual, and/or transgender (LGBT).

Objective

The purpose of this study was to evaluate pain screening and reported pain of LGBT Veterans compared to non-LGBT Veterans.

Methods

Using a retrospective cross-sectional cohort, data from the Corporate Data Warehouse, a national repository with clinical/administrative data, were analyzed. Veterans were classified as LGBT using natural language processing. We used a robust Poisson model to examine the association between LGBT status and binary outcomes of pain screening, any pain, and persistent pain within one year of entry in the cohort. All models were adjusted for demographics, mental health, substance use, musculoskeletal disorder(s), and number of clinic visits.

Results

There were 1,149,486 Veterans (218,154 (19%) classified as LGBT) in our study. Among LGBT Veterans, 94% were screened for pain compared to 89% among those not classified as LGBT (non-LGBT) Veterans. In adjusted models, LGBT Veterans' probability of being screened for pain compared to non-LGBT Veterans was 2.5% higher (95% CI 2.3%, 2.6%); risk of any pain was 2.1% lower (95% CI 1.6%, 2.6%); and there was no significant difference between LGBT and non-LGBT Veterans in persistent pain (RR = 1.00, 95% CI (0.99, 1.01), p = 0.88).

Conclusions

In a nationwide sample, LGBT Veterans were more likely to be screened for pain but had lower self-reported pain scores, though adjusted differences were small. It was notable that transgender and Black Veterans reported the greatest pain. Reasons for these findings require further investigation.

TransformEHR: transformer-based encoder-decoder generative model to enhance prediction of disease outcomes using electronic health records

Deep learning transformer-based models using longitudinal electronic health records (EHRs) have shown a great success in prediction of clinical diseases or outcomes. Pretraining on a large dataset can help such models map the input space better and boost their performance on relevant tasks through finetuning with limited data. In this study, we present TransformEHR, a generative encoder-decoder model with transformer that is pretrained using a new pretraining objective-predicting all diseases and outcomes of a patient at a future visit from previous visits. TransformEHR's encoder-decoder framework, paired with the novel pretraining objective, helps it achieve the new state-of-the-art performance on multiple clinical prediction tasks. Comparing with the previous model, TransformEHR improves area under the precision-recall curve by 2% (p < 0.001) for pancreatic cancer onset and by 24% (p = 0.007) for intentional self-harm in patients with post-traumatic stress disorder. The high performance in predicting intentional self-harm shows the potential of TransformEHR in building effective clinical intervention systems. TransformEHR is also generalizable and can be easily finetuned for clinical prediction tasks with limited data.

A Stakeholder-Informed Ethical Framework to Guide Implementation of Suicide Risk Prediction Models Derived from Electronic Health Records

OBJECTIVE

Develop a stakeholder-informed ethical framework to provide practical guidance to health systems considering implementation of suicide risk prediction models.

METHODS

In this multi-method study, patients and family members participating in formative focus groups (n = 4 focus groups, 23 participants), patient advisors, and a bioethics consultant collectively informed the development of a web-based survey; survey results (n = 1,357 respondents) and themes from interviews with stakeholders (patients, health system administrators, clinicians, suicide risk model developers, and a bioethicist) were used to draft the ethical framework.

RESULTS

Clinical, ethical, operational, and technical issues reiterated by multiple stakeholder groups and corresponding questions for risk prediction model adopters to consider prior to and during suicide risk model implementation are organized within six ethical principles in the resulting stakeholder-informed framework. Key themes include: patients' rights to informed consent and choice to conceal or reveal risk (autonomy); appropriate application of risk models, data and model limitations and consequences including ambiguous risk predictors in opaque models (explainability); selecting actionable risk thresholds (beneficence, distributive justice); access to risk information and stigma (privacy); unanticipated harms (non-maleficence); and planning for expertise and resources to continuously audit models, monitor harms, and redress grievances (stewardship).

CONCLUSIONS

Enthusiasm for risk prediction in the context of suicide is understandable given the escalating suicide rate in the U.S. Attention to ethical and practical concerns in advance of automated suicide risk prediction model implementation may help avoid unnecessary harms that could thwart the promise of this innovation in suicide prevention. HIGHLIGHTSPatients' desire to consent/opt out of suicide risk prediction models.Recursive ethical questioning should occur throughout risk model implementation.Risk modeling resources are needed to continuously audit models and monitor harms.

Evaluation of a Model to Target High-risk Psychiatric Inpatients for an Intensive Postdischarge Suicide Prevention Intervention

Importance

The months after psychiatric hospital discharge are a time of high risk for suicide. Intensive postdischarge case management, although potentially effective in suicide prevention, is likely to be cost-effective only if targeted at high-risk patients. A previously developed machine learning (ML) model showed that postdischarge suicides can be predicted from electronic health records and geospatial data, but it is unknown if prediction could be improved by adding additional information.

Objective

To determine whether model prediction could be improved by adding information extracted from clinical notes and public records.

Design, Setting, and Participants

Models were trained to predict suicides in the 12 months after Veterans Health Administration (VHA) short-term (less than 365 days) psychiatric hospitalizations between the beginning of 2010 and September 1, 2012 (299 050 hospitalizations, with 916 hospitalizations followed within 12 months by suicides) and tested in the hospitalizations from September 2, 2012, to December 31, 2013 (149 738 hospitalizations, with 393 hospitalizations followed within 12 months by suicides). Validation focused on net benefit across a range of plausible decision thresholds. Predictor importance was assessed with Shapley additive explanations (SHAP) values. Data were analyzed from January to August 2022.

Main Outcomes and Measures

Suicides were defined by the National Death Index. Base model predictors included VHA electronic health records and patient residential data. The expanded predictors came from natural language processing (NLP) of clinical notes and a social determinants of health (SDOH) public records database.

Results

The model included 448 788 unique hospitalizations. Net benefit over risk horizons between 3 and 12 months was generally highest for the model that included both NLP and SDOH predictors (area under the receiver operating characteristic curve range, 0.747-0.780; area under the precision recall curve relative to the suicide rate range, 3.87-5.75). NLP and SDOH predictors also had the highest predictor class-level SHAP values (proportional SHAP = 64.0% and 49.3%, respectively), although the single highest positive variable-level SHAP value was for a count of medications classified by the US Food and Drug Administration as increasing suicide risk prescribed the year before hospitalization (proportional SHAP = 15.0%).

Conclusions and Relevance

In this study, clinical notes and public records were found to improve ML model prediction of suicide after psychiatric hospitalization. The model had positive net benefit over 3-month to 12-month risk horizons for plausible decision thresholds. Although caution is needed in inferring causality based on predictor importance, several key predictors have potential intervention implications that should be investigated in future studies.

Associations Between Natural Language Processing-Enriched Social Determinants of Health and Suicide Death Among US Veterans

Importance

Social determinants of health (SDOHs) are known to be associated with increased risk of suicidal behaviors, but few studies use SDOHs from unstructured electronic health record notes.

Objective

To investigate associations between veterans' death by suicide and recent SDOHs, identified using structured and unstructured data.

Design, Setting, and Participants

This nested case-control study included veterans who received care under the US Veterans Health Administration from October 1, 2010, to September 30, 2015. A natural language processing (NLP) system was developed to extract SDOHs from unstructured clinical notes. Structured data yielded 6 SDOHs (ie, social or familial problems, employment or financial problems, housing instability, legal problems, violence, and nonspecific psychosocial needs), NLP on unstructured data yielded 8 SDOHs (social isolation, job or financial insecurity, housing instability, legal problems, barriers to care, violence, transition of care, and food insecurity), and combining them yielded 9 SDOHs. Data were analyzed in May 2022.

Exposures

Occurrence of SDOHs over a maximum span of 2 years compared with no occurrence of SDOH.

Main Outcomes and Measures

Cases of suicide death were matched with 4 controls on birth year, cohort entry date, sex, and duration of follow-up. Suicide was ascertained by National Death Index, and patients were followed up for up to 2 years after cohort entry with a study end date of September 30, 2015. Adjusted odds ratios (aORs) and 95% CIs were estimated using conditional logistic regression.

Results

Of 6 122 785 veterans, 8821 committed suicide during 23 725 382 person-years of follow-up (incidence rate 37.18 per 100 000 person-years). These 8821 veterans were matched with 35 284 control participants. The cohort was mostly male (42 540 [96.45%]) and White (34 930 [79.20%]), with 6227 (14.12%) Black veterans. The mean (SD) age was 58.64 (17.41) years. Across the 5 common SDOHs, NLP-extracted SDOH, on average, retained 49.92% of structured SDOHs and covered 80.03% of all SDOH occurrences. SDOHs, obtained by structured data and/or NLP, were significantly associated with increased risk of suicide. The 3 SDOHs with the largest effect sizes were legal problems (aOR, 2.66; 95% CI, 2.46-2.89), violence (aOR, 2.12; 95% CI, 1.98-2.27), and nonspecific psychosocial needs (aOR, 2.07; 95% CI, 1.92-2.23), when obtained by combining structured data and NLP.

Conclusions and Relevance

In this study, NLP-extracted SDOHs, with and without structured SDOHs, were associated with increased risk of suicide among veterans, suggesting the potential utility of NLP in public health studies.

Using Machine Learning to Examine Suicidal Ideation After Traumatic Brain Injury: A Traumatic Brain Injury Model Systems National Database Study

OBJECTIVE

The aim of the study was to predict suicidal ideation 1 yr after moderate to severe traumatic brain injury.

DESIGN

This study used a cross-sectional design with data collected through the prospective, longitudinal Traumatic Brain Injury Model Systems network at hospitalization and 1 yr after injury. Participants who completed the Patient Health Questionnaire-9 suicide item at year 1 follow-up ( N = 4328) were included.

RESULTS

A gradient boosting machine algorithm demonstrated the best performance in predicting suicidal ideation 1 yr after traumatic brain injury. Predictors were Patient Health Questionnaire-9 items (except suicidality), Generalized Anxiety Disorder-7 items, and a measure of heavy drinking. Results of the 10-fold cross-validation gradient boosting machine analysis indicated excellent classification performance with an area under the curve of 0.882. Sensitivity was 0.85 and specificity was 0.77. Accuracy was 0.78 (95% confidence interval, 0.77-0.79). Feature importance analyses revealed that depressed mood and guilt were the most important predictors of suicidal ideation, followed by anhedonia, concentration difficulties, and psychomotor disturbance.

CONCLUSIONS

Overall, depression symptoms were most predictive of suicidal ideation. Despite the limited clinical impact of the present findings, machine learning has potential to improve prediction of suicidal behavior, leveraging electronic health record data, to identify individuals at greatest risk, thereby facilitating intervention and optimization of long-term outcomes after traumatic brain injury.

Identifying Youth at Risk for Suicidal Thoughts and Behaviors Using the "p" factor in Primary Care: An Exploratory Study

Suicide is a major, preventable public health problem. The general factor of psychopathology ("p" factor) might help improve detection and prediction of individuals at risk for suicide. This cross-sectional proof-of-concept study tests whether the p-factor score is associated with suicidal thoughts and behaviors (STB) better than a depression scale alone. Youth (N = 841; mean age 18.02, SD = 3.36) in primary care were universally screened using the Behavioral Health Screen (BHS). Factor analysis and ROC results showed the BHS assesses the p-factor, and the p-factor score demonstrates higher classification accuracy of several types of STB than a depression scale. The p-factor could help clinicians in the identification of youths with STB.

Structured data vs. unstructured data in machine learning prediction models for suicidal behaviors: A systematic review and meta-analysis

Suicide remains a leading cause of preventable death worldwide, despite advances in research and decreases in mental health stigma through government health campaigns. Machine learning (ML), a type of artificial intelligence (AI), is the use of algorithms to simulate and imitate human cognition. Given the lack of improvement in clinician-based suicide prediction over time, advancements in technology have allowed for novel approaches to predicting suicide risk. This systematic review and meta-analysis aimed to synthesize current research regarding data sources in ML prediction of suicide risk, incorporating and comparing outcomes between structured data (human interpretable such as psychometric instruments) and unstructured data (only machine interpretable such as electronic health records). Online databases and gray literature were searched for studies relating to ML and suicide risk prediction. There were 31 eligible studies. The outcome for all studies combined was AUC = 0.860, structured data showed AUC = 0.873, and unstructured data was calculated at AUC = 0.866. There was substantial heterogeneity between the studies, the sources of which were unable to be defined. The studies showed good accuracy levels in the prediction of suicide risk behavior overall. Structured data and unstructured data also showed similar outcome accuracy according to meta-analysis, despite different volumes and types of input data.

Predicting suicide attempts among U.S. Army soldiers after leaving active duty using information available before leaving active duty: results from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS)

Suicide risk is elevated among military service members who recently transitioned to civilian life. Identifying high-risk service members before this transition could facilitate provision of targeted preventive interventions. We investigated the feasibility of doing this by attempting to develop a prediction model for self-reported suicide attempts (SAs) after leaving or being released from active duty in the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS). This study included two self-report panel surveys (LS1: 2016-2018, LS2: 2018-2019) administered to respondents who previously participated while on active duty in one of three Army STARRS 2011-2014 baseline self-report surveys. We focus on respondents who left active duty >12 months before their LS survey (n = 8899). An ensemble machine learning model using predictors available prior to leaving active duty was developed in a 70% training sample and validated in a 30% test sample. The 12-month self-reported SA prevalence (SE) was 1.0% (0.1). Test sample AUC (SE) was 0.74 (0.06). The 15% of respondents with highest predicted risk included nearly two-thirds of 12-month SAs and over 80% of medically serious 12-month SAs. These results show that it is possible to identify soldiers at high post-transition self-report SA risk before the transition. Future model development is needed to examine prediction of SAs assessed by administrative data and using surveys administered closer to the time of leaving active duty.

Translating promise into practice: a review of machine learning in suicide research and prevention

In ever more pressured health-care systems, technological solutions offering scalability of care and better resource targeting are appealing. Research on machine learning as a technique for identifying individuals at risk of suicidal ideation, suicide attempts, and death has grown rapidly. This research often places great emphasis on the promise of machine learning for preventing suicide, but overlooks the practical, clinical implementation issues that might preclude delivering on such a promise. In this Review, we synthesise the broad empirical and review literature on electronic health record-based machine learning in suicide research, and focus on matters of crucial importance for implementation of machine learning in clinical practice. The challenge of preventing statistically rare outcomes is well known; progress requires tackling data quality, transparency, and ethical issues. In the future, machine learning models might be explored as methods to enable targeting of interventions to specific individuals depending upon their level of need-ie, for precision medicine. Primarily, however, the promise of machine learning for suicide prevention is limited by the scarcity of high-quality scalable interventions available to individuals identified by machine learning as being at risk of suicide.

Artificial intelligence and suicide prevention: a systematic review

Background

Suicide is one of the main preventable causes of death. Artificial intelligence (AI) could improve methods for assessing suicide risk. The objective of this review is to assess the potential of AI in identifying patients who are at risk of attempting suicide.

Methods

A systematic review of the literature was conducted on PubMed, EMBASE, and SCOPUS databases, using relevant keywords.

Results

Thanks to this research, 296 studies were identified. Seventeen studies, published between 2014 and 2020 and matching inclusion criteria, were selected as relevant. Included studies aimed at predicting individual suicide risk or identifying at-risk individuals in a specific population. The AI performance was overall good, although variable across different algorithms and application settings.

Conclusions

AI appears to have a high potential for identifying patients at risk of suicide. The precise use of these algorithms in clinical situations, as well as the ethical issues it raises, remain to be clarified.

Suicide typologies among Medicaid beneficiaries, North Carolina 2014-2017

BACKGROUND

There is a well-established need for population-based screening strategies to identify people at risk of suicide. Because only about half of suicide decedents are ever diagnosed with a behavioral health condition, it may be necessary for providers to consider life circumstances that may also put individuals at risk. This study described the alignment of medical diagnoses with life circumstances by identifying suicide typologies among decedents. Demographics, stressful life events, suicidal behavior, perceived and diagnosed health problems, and suicide method contributed to the typologies.

METHODS

This study linked North Carolina Medicaid and North Carolina Violent Death Reporting System (NC-VDRS) data for analysis in 2020. For suicide decedents from 2014 to 2017 aged 25-54 years, we analyzed 12 indicators of life circumstances from NC-VDRS and 6 indicators from Medicaid claims, using a latent class model. Separate models were developed for men and women.

RESULTS

Most decedents were White (88.3%), with a median age of 41, and over 70% had a health care visit in the 90 days prior to suicide. Two typologies were identified in both males (n = 175) and females (n = 153). Both typologies had similar profiles of life circumstances, but one had high probabilities of diagnosed behavioral health conditions (45% of men, 71% of women), compared to low probabilities in the other (55% of men, 29% of women). Black beneficiaries and men who died by firearm were over-represented in the less-diagnosed class, though estimates were imprecise (odds ratio for Black men: 3.1, 95% confidence interval: 0.8, 12.4; odds ratio for Black women: 5.0, 95% confidence interval: 0.9, 31.2; odds ratio for male firearm decedents: 1.6, 95% confidence interval: 0.7, 3.4).

CONCLUSIONS

Nearly half of suicide decedents have a typology characterized by low probability of diagnosis of behavioral health conditions. Suicide screening could likely be enhanced using improved indicators of lived experience and behavioral health.

Artificial Intelligence: Review of Current and Future Applications in Medicine

Background

The role of artificial intelligence (AI) in health care is expanding rapidly. Currently, there are at least 29 US Food and Drug Administration-approved AI health care devices that apply to numerous medical specialties and many more are in development.

Observations

With increasing expectations for all health care sectors to deliver timely, fiscally-responsible, high-quality health care, AI has potential utility in numerous areas, such as image analysis, improved workflow and efficiency, public health, and epidemiology, to aid in processing large volumes of patient and medical data. In this review, we describe basic terminology, principles, and general AI applications relating to health care. We then discuss current and future applications for a variety of medical specialties. Finally, we discuss the future potential of AI along with the potential risks and limitations of current AI technology.

Conclusions

AI can improve diagnostic accuracy, increase patient safety, assist with patient triage, monitor disease progression, and assist with treatment decisions.

Leveraging data science to enhance suicide prevention research: a literature review

OBJECTIVE

The purpose of this research is to identify how data science is applied in suicide prevention literature, describe the current landscape of this literature and highlight areas where data science may be useful for future injury prevention research.

DESIGN

We conducted a literature review of injury prevention and data science in April 2020 and January 2021 in three databases.

METHODS

For the included 99 articles, we extracted the following: (1) author(s) and year; (2) title; (3) study approach (4) reason for applying data science method; (5) data science method type; (6) study description; (7) data source and (8) focus on a disproportionately affected population.

RESULTS

Results showed the literature on data science and suicide more than doubled from 2019 to 2020, with articles with individual-level approaches more prevalent than population-level approaches. Most population-level articles applied data science methods to describe (n=10) outcomes, while most individual-level articles identified risk factors (n=27). Machine learning was the most common data science method applied in the studies (n=48). A wide array of data sources was used for suicide research, with most articles (n=45) using social media and web-based behaviour data. Eleven studies demonstrated the value of applying data science to suicide prevention literature for disproportionately affected groups.

CONCLUSION

Data science techniques proved to be effective tools in describing suicidal thoughts or behaviour, identifying individual risk factors and predicting outcomes. Future research should focus on identifying how data science can be applied in other injury-related topics.

Invited Commentary: New Directions in Machine Learning Analyses of Administrative Data to Prevent Suicide-Related Behaviors

This issue contains a thoughtful report by Gradus et al. (Am J Epidemiol. 2021;190(12):2517-2527) on a machine learning analysis of administrative variables to predict suicide attempts over 2 decades throughout Denmark. This is one of numerous recent studies that document strong concentration of risk of suicide-related behaviors among patients with high scores on machine learning models. The clear exposition of Gradus et al. provides an opportunity to review major challenges in developing, interpreting, and using such models: defining appropriate controls and time horizons, selecting comprehensive predictors, dealing with imbalanced outcomes, choosing classifiers, tuning hyperparameters, evaluating predictor variable importance, and evaluating operating characteristics. We close by calling for machine-learning research into suicide-related behaviors to move beyond merely demonstrating significant prediction-this is by now well-established-and to focus instead on using such models to target specific preventive interventions and to develop individualized treatment rules that can be used to help guide clinical decisions to address the growing problems of suicide attempts, suicide deaths, and other injuries and deaths in the same spectrum.

Risk Factors Associated With Nonfatal Opioid Overdose Leading to Intensive Care Unit Admission: A Cross-sectional Study

BACKGROUND

Opioid overdose (OD) and related deaths have significantly increased in the United States over the last 2 decades. Existing studies have mostly focused on demographic and clinical risk factors in noncritical care settings. Social and behavioral determinants of health (SBDH) are infrequently coded in the electronic health record (EHR) and usually buried in unstructured EHR notes, reflecting possible gaps in clinical care and observational research. Therefore, SBDH often receive less attention despite being important risk factors for OD. Natural language processing (NLP) can alleviate this problem.

OBJECTIVE

The objectives of this study were two-fold: First, we examined the usefulness of NLP for SBDH extraction from unstructured EHR text, and second, for intensive care unit (ICU) admissions, we investigated risk factors including SBDH for nonfatal OD.

METHODS

We performed a cross-sectional analysis of admission data from the EHR of patients in the ICU of Beth Israel Deaconess Medical Center between 2001 and 2012. We used patient admission data and International Classification of Diseases, Ninth Revision (ICD-9) diagnoses to extract demographics, nonfatal OD, SBDH, and other clinical variables. In addition to obtaining SBDH information from the ICD codes, an NLP model was developed to extract 6 SBDH variables from EHR notes, namely, housing insecurity, unemployment, social isolation, alcohol use, smoking, and illicit drug use. We adopted a sequential forward selection process to select relevant clinical variables. Multivariable logistic regression analysis was used to evaluate the associations with nonfatal OD, and relative risks were quantified as covariate-adjusted odds ratios (aOR).

RESULTS

The strongest association with nonfatal OD was found to be drug use disorder (aOR 8.17, 95% CI 5.44-12.27), followed by bipolar disorder (aOR 2.69, 95% CI 1.68-4.29). Among others, major depressive disorder (aOR 2.57, 95% CI 1.12-5.88), being on a Medicaid health insurance program (aOR 2.26, 95% CI 1.43-3.58), history of illicit drug use (aOR 2.09, 95% CI 1.15-3.79), and current use of illicit drugs (aOR 2.06, 95% CI 1.20-3.55) were strongly associated with increased risk of nonfatal OD. Conversely, Blacks (aOR 0.51, 95% CI 0.28-0.94), older age groups (40-64 years: aOR 0.65, 95% CI 0.44-0.96; >64 years: aOR 0.16, 95% CI 0.08-0.34) and those with tobacco use disorder (aOR 0.53, 95% CI 0.32-0.89) or alcohol use disorder (aOR 0.64, 95% CI 0.42-1.00) had decreased risk of nonfatal OD. Moreover, 99.82% of all SBDH information was identified by the NLP model, in contrast to only 0.18% identified by the ICD codes.

CONCLUSIONS

This is the first study to analyze the risk factors for nonfatal OD in an ICU setting using NLP-extracted SBDH from EHR notes. We found several risk factors associated with nonfatal OD including SBDH. SBDH are richly described in EHR notes, supporting the importance of integrating NLP-derived SBDH into OD risk assessment. More studies in ICU settings can help health care systems better understand and respond to the opioid epidemic.

Using machine learning to predict suicide in the 30 days after discharge from psychiatric hospital in Denmark

BACKGROUND

Suicide risk is high in the 30 days after discharge from psychiatric hospital, but knowledge of the profiles of high-risk patients remains limited.

AIMS

To examine sex-specific risk profiles for suicide in the 30 days after discharge from psychiatric hospital, using machine learning and Danish registry data.

METHOD

We conducted a case-cohort study capturing all suicide cases occurring in the 30 days after psychiatric hospital discharge in Denmark from 1 January 1995 to 31 December 2015 (n = 1205). The comparison subcohort was a 5% random sample of all persons born or residing in Denmark on 1 January 1995, and who had a first psychiatric hospital admission between 1995 and 2015 (n = 24 559). Predictors included diagnoses, surgeries, prescribed medications and demographic information. The outcome was suicide death recorded in the Danish Cause of Death Registry.

RESULTS

For men, prescriptions for anxiolytics and drugs used in addictive disorders interacted with other characteristics in the risk profiles (e.g. alcohol-related disorders, hypnotics and sedatives) that led to higher risk of postdischarge suicide. In women, there was interaction between recurrent major depression and other characteristics (e.g. poisoning, low income) that led to increased risk of suicide. Random forests identified important suicide predictors: alcohol-related disorders and nicotine dependence in men and poisoning in women.

CONCLUSIONS

Our findings suggest that accurate prediction of suicide during the high-risk period immediately after psychiatric hospital discharge may require a complex evaluation of multiple factors for men and women.

Meta-analysis of the strength of exploratory suicide prediction models; from clinicians to computers

BACKGROUND

Suicide prediction models have been formulated in a variety of ways and are heterogeneous in the strength of their predictions. Machine learning has been a proposed as a way of improving suicide predictions by incorporating more suicide risk factors.

AIMS

To determine whether machine learning and the number of suicide risk factors included in suicide prediction models are associated with the strength of the resulting predictions.

METHOD

Random-effect meta-analysis of exploratory suicide prediction models constructed by combining two or more suicide risk factors or using clinical judgement (Prospero Registration CRD42017059665). Studies were located by searching for papers indexed in PubMed before 15 August 2020 with the term suicid* in the title.

RESULTS

In total, 86 papers reported 102 suicide prediction models and included 20 210 411 people and 106 902 suicides. The pooled odds ratio was 7.7 (95% CI 6.7-8.8) with high between-study heterogeneity (I2 = 99.5). Machine learning was associated with a non-significantly higher odds ratio of 11.6 (95% CI 6.0-22.3) and clinical judgement with a non-significantly lower odds ratio of 4.7 (95% CI 2.1-10.9). Models including a larger number of suicide risk factors had a higher odds ratio when machine-learning studies were included (P = 0.02). Among non-machine-learning studies, suicide prediction models including fewer risk factors performed just as well as those including more risk factors.

CONCLUSIONS

Machine learning might have the potential to improve the performance of suicide prediction models by increasing the number of included suicide risk factors but its superiority over other methods is unproven.

Areas of Improvement in Suicide Risk Identification, Assessment, and Risk Mitigation Documentation by Mental Health Prescribers at a Veterans Affairs Health Care System

PURPOSE

Veterans have a suicide rate 1.5 times higher than the non-veteran population. The Department of Veterans Affairs (VA) implemented suicide risk screening recommendations in 2018. This project assessed the impact of mental health (MH) prescribers on these recommendations and identified areas of improvement.

METHODS

Seventy MH Clinical Pharmacy Specialists (CPS) and 52 other MH prescribers were included. Patients with a positive question nine (from the Patient Health Questionnaire-9) and a same-day Columbia Suicide Severity Rating Scale (C-SSRS) between 11/01/18 and 11/01/19 at a VA system were reviewed. Completion of same-day Comprehensive Suicide Risk Evaluation (CSRE), same-day Suicide Prevention Safety Plan (SPSP), number of patients who were not offered naloxone despite access to opioids, and number of patients who were not offered a gunlock despite access to firearms were compared between groups. Time from C-SSRS to suicidal behavior was compared between those who did and did not receive a same-day CSRE.

RESULTS

MH CPS were significantly more likely to complete a same-day CSRE (p = 0.0201) and SPSP (p < 0.001) when recommended. Naloxone outcomes were not assessed due to availability of only one data point. Rates of gunlock offers did not differ significantly between groups (Fisher's exact test, p = 0.3189) however there was no documentation stating why they were not offered when appropriate 40% of the time. Time to suicidal behavior did not vary across patients based on CSRE completion (p = 0.16).

CONCLUSION

MH CPS play an important role in suicide risk screening for veterans. There is a need for improvement regarding the offering and documentation of firearm risk mitigation strategies.