Screening and Assessing Suicide Risk in Medical Settings: Feasible Strategies for Early Detection

Clinician Suicide Risk Assessment for Prediction of Suicide Attempt in a Large Health Care System

Importance

Clinical practice guidelines recommend suicide risk screening and assessment across behavioral health settings. The predictive accuracy of real-world clinician assessments for stratifying patients by risk of future suicidal behavior, however, remains understudied.

Objective

To evaluate routine clinical suicide risk assessment for prospectively predicting suicide attempt.

Design, Setting, and Participants

This electronic health record-based, prognostic study included 89 957 patients (≥5 years of age) with a structured suicide risk assessment (based on the Suicide Assessment Five-step Evaluation and Triage framework) that was documented by 2577 clinicians during outpatient, inpatient, and emergency department encounters at 12 hospitals in the Mass General Brigham health system between July 2019 and February 2023.

Main Outcomes and Measures

The primary outcome was an emergency department visit with a suicide attempt code recorded in the electronic health record within 90 days or 180 days of the index suicide risk assessment. The predictive performance of suicide risk assessments was evaluated on a temporal test set first using stratified prevalence (clinicians' overall risk estimates from a single suicide risk assessment item indicating minimal, low, moderate, or high risk) and then using machine learning models (incorporating all suicide risk assessment items).

Results

Of the 812 114 analyzed suicide risk assessments from the electronic health record, 58.81% were with female patients and 3.27% were with patients who were Asian, 5.26% were Black, 3.02% were Hispanic, 77.44% were White, and 11.00% were of Other or Unknown race. After suicide risk assessments were conducted during outpatient encounters, the suicide attempt rate was 0.12% within 90 days and 0.22% within 180 days; for inpatient encounters, the rate was 0.79% within 90 days and 1.29% within 180 days; and for emergency department encounters, the rate was 2.40% within 90 days and 3.70% within 180 days. Among patients evaluated during outpatient encounters, clinicians' overall single-item risk estimates had an area under the curve (AUC) value of 0.77 (95% CI, 0.72-0.81) for 90-day suicide attempt prediction; among patients evaluated during inpatient encounters, the AUC was 0.64 (95% CI, 0.59-0.69); and among patients evaluated during emergency department encounters, the AUC was 0.60 (95% CI, 0.55-0.64). Incorporating all clinician-documented suicide risk assessment items (87 predictors) via machine learning significantly increased the AUC for 90-day risk prediction to 0.87 (95% CI, 0.83-0.90) among patients evaluated during outpatient encounters, 0.79 (95% CI, 0.74-0.84) among patients evaluated during inpatient encounters, and 0.76 (95% CI, 0.72-0.80) among patients evaluated during emergency department encounters. Performance was similar for 180-day suicide risk prediction. The positive predictive values for the best-performing machine learning models (with 95% specificity) ranged from 3.6 to 10.1 times the prevalence for suicide attempt.

Conclusions and Relevance

Clinicians stratify patients for suicide risk at levels significantly above chance. However, the predictive accuracy improves significantly by statistically incorporating information about recent suicidal thoughts and behaviors and other factors routinely assessed during clinical suicide risk assessment.

Effectiveness of generative AI-large language models' recognition of veteran suicide risk: a comparison with human mental health providers using a risk stratification model

Background

With over 6,300 United States military veterans dying by suicide annually, the Veterans Health Administration (VHA) is exploring innovative strategies, including artificial intelligence (AI), for suicide risk assessment. Machine learning has been predominantly utilized, but the application of generative AI-large language models (GAI-LLMs) remains unexplored.

Objective

This study evaluates the effectiveness of GAI-LLMs, specifically ChatGPT-3.5, ChatGPT-4o, and Google Gemini, in using the VHA's Risk Stratification Table for identifying suicide risks and making treatment recommendations in response to standardized veteran vignettes.

Methods

We compared the GAI-LLMs' assessments and recommendations for both acute and chronic suicide risks to evaluations by mental health care providers (MHCPs). Four vignettes, representing varying levels of suicide risk, were used.

Results

GAI-LLMs' assessments showed discrepancies with MHCPs, particularly rating the most acute case as less acute and the least acute case as more acute. For chronic risk, GAI-LLMs' evaluations were generally in line with MHCPs, except for one vignette rated with higher chronic risk by the GAI-LLM. Variation across GAI-LLMs was also observed. Notably, ChatGPT-3.5 showed lower acute risk ratings compared to ChatGPT-4o and Google Gemini, while ChatGPT-4o identified higher chronic risk ratings and recommended hospitalization for all veterans. Treatment planning by GAI-LLMs was predicted by chronic but not acute risk ratings.

Conclusion

While GAI-LLMs offers potential suicide risk assessment comparable to MHCPs, significant variation exists across different GAI-LLMs in both risk evaluation and treatment recommendations. Continued MHCP oversight is essential to ensure accuracy and appropriate care.

Implications

These findings highlight the need for further research into optimizing GAI-LLMs for consistent and reliable use in clinical settings, ensuring they complement rather than replace human expertise.

Geographic Patterns of Youth Suicide in San Diego County

OBJECTIVE

Our objectives were to evaluate for any cluster patterns of youth suicide deaths and characterize the level of child opportunity in the communities where suicide deaths occurred.

METHODS

Decedents <18 years were identified from San Diego County Medical Examiner death reports from 2000 to 2020. We mapped decedents' residential Zone Improvement Plan (ZIP) codes and calculated suicide rates per 10,000 youths. ZIP codes identified in overlapping spatial statistical approaches - the spatial scan statistic and Local Moran with Empirical Bayes (EB) rates - were considered a cluster for the final analysis. We obtained Child Opportunity Index (COI) scores for each ZIP code to determine if there were differences in: 1) ZIP codes with suicide deaths compared to ZIPs with no deaths 2) differences in distribution of suicide death rates across quintiles of COI.

RESULTS

Scan statistic identified 25 ZIP codes within a cluster (RR 2.6, P = 0.00066). Local Moran with EB rates identified two ZIP codes as a high-high cluster (P < 0.05). The location identified as a cluster in both approaches was in Alpine. The median COI for ZIP codes with suicide deaths was higher at 63.5 (IQR 38-83) compared to 47 (IQR 22.5-75.5) for ZIP codes without suicide deaths. There was a significant difference in suicide rates between Very Low and Moderate levels of Overall opportunity (P = .013).

CONCLUSION

We identified a cluster of youth suicides in one of the most populous counties in the country. These findings serve to inform policies and prevention programs that aim to mitigate youth suicide mortality.

Temporal Patterns of Suicidality Among Adolescents Receiving Behavioral and Mental Health Services in the Community: A Survival Analysis

Adolescent suicide is a major public health concern, particularly among adolescents who have endured Adverse Childhood Experiences (ACEs). Adolescents who have been exposed to multiple ACEs are as much as three times more likely to present with suicidality compared to the general adolescent population. Adolescents who have been exposed to multiple ACEs are also more likely to receive behavioral and mental health services in the community. It is therefore important to understand patterns of suicidality among this sub-population of adolescents in order to provide the best clinical care. The present study examined the temporal patterns of suicidality among adolescents who have been exposed to multiple ACEs and are receiving behavioral and mental health services in the community. Using Electronic Health Record (EHR) data from a community-based behavioral and mental health care organization, an exploratory survival analysis was conducted on time to suicidal thoughts and behaviors (STBs) after suicidality risk screen at intake. Average time from suicidality risk screen at intake to STB was 185 days (6.2 months). Youth who screened negative for suicidality risk at intake had a longer survival time than youth who screened positive for suicidality risk, and the survival distributions between the two groups was significant. Predictors of STBs were also examined, with gender being a significant predictor of an STB occurring during the follow-up period. These findings may be used to guide suicidality screening and clinical practice at community-based behavioral and mental health care organizations serving adolescents who have been exposed to multiple ACEs.

Measurement Choices for Youth Suicidality

Suicide is among the leading causes of death among individuals ages 10-24, making suicidal thoughts and behaviors (STBs) a serious public health crisis among youth. Suicide risk screening and assessment are vital to addressing this public health crisis. In fact, many youths that screen positive for suicidal ideation do not have known mental health concerns and would have been missed if not asked directly. Medical settings are an optimal setting to detect suicidality early and provide appropriate follow-up monitoring and care as needed. To support effective and efficient screening and assessment of suicidal thoughts and behaviors, providers must choose measures with both strong psychometric properties and clinical utility. While measurement of STBs can vary across health settings, suicide risk screening and assessment typically involves gathering information about current suicidal ideation, suicidal behaviors, and suicidal plans via self-report questionnaires, clinical interviews, and/or computerized adaptive screens. In alignment with measurement-based care efforts, the current manuscript will provide a scoping review of measures of youth suicidal ideation, behavior, plans, and their risk factors. Specifically, the psychometric properties, clinical utility, and other key considerations for screening and assessment of adolescent suicide risk are discussed.