Comparing the predictive value of screening to the use of electronic health record data for detecting future suicidal thoughts and behavior in an urban pediatric emergency department: A preliminary analysis

Predictive Validity of the K-CAT-SS in High-Risk Adolescents and Young Adults

OBJECTIVE

Suicide is a leading cause of death in adolescents and young adults and has increased substantially in the past 15 years. Accurate suicide risk stratification based on rapid screening can help reverse these trends. To study the ability of the K-CAT-SS, a brief computerized adaptive test of suicidality, to predict suicide attempts (SA) in high-risk youth.

METHOD

652 participants aged 12-24, 78% of whom presented with suicidal ideation or behavior, were recruited within one month of mental health contact. The K-CAT-SS, scaled from 0-100, was administered at baseline, and participants were assessed at around 1-, 3-, and 6-months post-intake. Weekly incidence of SAs, assessed using the Adolescent Longitudinal Interval Follow-up Evaluation and Columbia-Suicide Severity Rating Scale. A secondary outcome was suicidal behavior (SB), including aborted, interrupted, and actual SA.

RESULTS

The K-CAT-SS showed a 4.91-fold increase in SAs for every 25-point increase in the baseline score, (95% CI=2.83,8.52) and a 3.51-fold increase in SBs (95% CI=2.32,5.30) These relationships persisted following adjustment for prior attempts, and demographic variables including age, sex, gender identity, sexual orientation, race/ethnicity, and other measures of psychopathology. No moderating effects were identified. At 3 months, the Area Under the Receiver Operating Curve [AUROC]=0.83 (95% CI=0.72, 0.93) for 1 or more SA.

CONCLUSION

The K-CAT-SS is an excellent tool for suicide risk stratification, particularly in higher-risk populations where other measures have shown lower predictive validity.

Adolescent, Parent, and Provider Perceptions of a Predictive Algorithm to Identify Adolescent Suicide Risk in Primary Care

OBJECTIVE

To understand adolescent, parent, and provider perceptions of a machine learning algorithm for detecting adolescent suicide risk prior to its implementation primary care.

METHODS

We conducted semi-structured, qualitative interviews with adolescents (n = 9), parents (n = 12), and providers (n = 10; mixture of behavioral health and primary care providers) across two major health systems. Interviews were audio recorded and transcribed with analyses supported by use of NVivo. A codebook was developed combining codes derived inductively from interview transcripts and deductively from implementation science frameworks for content analysis.

RESULTS

Reactions to the algorithm were mixed. While many participants expressed privacy concerns, they believed the algorithm could be clinically useful for identifying adolescents at risk for suicide and facilitating follow-up. Parents' past experiences with their adolescents' suicidal thoughts and behaviors contributed to their openness to the algorithm. Results also aligned with several key Consolidated Framework for Implementation Research domains. For example, providers mentioned barriers inherent to the primary care setting such as time and resource constraints likely to impact algorithm implementation. Participants also cited a climate of mistrust of science and health care as potential barriers.

CONCLUSIONS

Findings shed light on factors that warrant consideration to promote successful implementation of suicide predictive algorithms in pediatric primary care. By attending to perspectives of potential end users prior to the development and testing of the algorithm, we can ensure that the risk prediction methods will be well-suited to the providers who would be interacting with them and the families who could benefit.

Management of suicidal risk in the emergency department: A clinical pathway using the computerized adaptive screen for suicidal youth

Objective

Given the critical need for efficient and tailored suicide screening for youth presenting in the emergency department (ED), this study establishes validated screening score thresholds for the Computerized Adaptive Screen for Suicidal Youth (CASSY) and presents an example of a suicide risk classification pathway.

Methods

Participants were primarily from the Study One derivation cohort of the Emergency Department Screen for Teens at Risk for Suicide (ED-STARS) enrolled in collaboration with Pediatric Emergency Care Applied Research Networks (PECARN). CASSY scores corresponded to the predicted probabilities of a suicide attempt in the next 3 months and risk thresholds were classified as minimal (<1%), low (1%-5%), moderate (5%-10%), and high (>10%). CASSY scores were compared to risk thresholds derived from clinical consensus and ED complaints and dispositions. CASSY risk thresholds were also examined as predictors of future suicide attempts in the Study Two validation cohort of ED-STARS.

Results

A total of 1452 teens were enrolled with a median age of 15.2 years, 59.5% were female, 55.6% were White, 22% were Black, 22.3% were Latinx, and 42.8% received public assistance. The clinical consensus suicide risk groups were strongly associated with the CASSY-predicted risk thresholds. Suicide attempts in the Study Two cohort occurred at a frequency consistent with the CASSY-predicted thresholds.

Conclusions

The CASSY can be a valuable tool in providing patient-specific risk probabilities for a suicide attempt at 3 months and tailor the threshold cutoffs based on the availability of local mental health resources. We give an example of a clinical risk pathway, which should include segmentation of the ED population by medical versus psychiatric chief complaint.

Suicide Risk Screening Tools for Pediatric Patients: A Systematic Review of Test Accuracy

CONTEXT

Health care settings have increasingly adopted universal suicide risk screening tools into nonpsychiatric pediatric care; however, a systematic review examining the accuracy of these tools does not yet exist.

OBJECTIVE

Identify and review research on the test accuracy of suicide risk screening tools for pediatric patients in nonpsychiatric medical settings.

DATA SOURCES

PubMed and PsycINFO were searched to identify peer-reviewed articles published before March 23, 2023.

STUDY SELECTION

Articles that quantified the accuracy of a suicide risk screening tool (eg, sensitivity, specificity) in a nonpsychiatric medical setting (eg, primary care, specialty care, inpatient or surgical units, or the emergency department) were included.

DATA EXTRACTION

A total of 13 studies were included in this review. Screening tool psychometric properties and study risk of bias were evaluated.

RESULTS

Sensitivity among individual studies ranged from 50% to 100%, and specificity ranged from 58.8% to 96%. Methodological quality was relatively varied, and applicability concerns were low. When stratifying results by screening tool, the Ask Suicide-Screening Questions and Computerized Adaptive Screen for Suicidal Youth had the most robust evidence base.

LIMITATIONS

Because of considerable study heterogeneity, a meta-analytic approach was deemed inappropriate. This prevented us from statistically testing for differences between identified screening tools.

CONCLUSIONS

The Ask Suicide-Screening Questions and Computerized Adaptive Screen for Suicidal Youth exhibit satisfactory test accuracy and appear promising for integration into clinical practice. Although initial findings are promising, additional research targeted at examining the accuracy of screening tools among diverse populations is needed to ensure the equity of screening efforts.

Predicting suicide death after emergency department visits with mental health or self-harm diagnoses

OBJECTIVE

Use health records data to predict suicide death following emergency department visits.

METHODS

Electronic health records and insurance claims from seven health systems were used to: identify emergency department visits with mental health or self-harm diagnoses by members aged 11 or older; extract approximately 2500 potential predictors including demographic, historical, and baseline clinical characteristics; and ascertain subsequent deaths by self-harm. Logistic regression with lasso and random forest models predicted self-harm death over 90 days after each visit.

RESULTS

Records identified 2,069,170 eligible visits, 899 followed by suicide death within 90 days. The best-fitting logistic regression with lasso model yielded an area under the receiver operating curve of 0.823 (95% CI 0.810-0.836). Visits above the 95th percentile of predicted risk included 34.8% (95% CI 31.1-38.7) of subsequent suicide deaths and had a 0.303% (95% CI 0.261-0.346) suicide death rate over the following 90 days. Model performance was similar across subgroups defined by age, sex, race, and ethnicity.

CONCLUSIONS

Machine learning models using coded data from health records have moderate performance in predicting suicide death following emergency department visits for mental health or self-harm diagnosis and could be used to identify patients needing more systematic follow-up.

Suicide Risk Screening for Head and Neck Cancer Patients: An Implementation Study

OBJECTIVES

 There is limited research on suicide risk screening (SRS) among head and neck cancer (HNC) patients, a population at increased risk for suicide. To address this gap, this single-site mixed methods study assessed oncology professionals' perspectives about the feasibility, acceptability, and appropriateness of an electronic SRS program that was implemented as a part of routine care for HNC patients.

METHODS

Staff who assisted with SRS implementation completed (e.g., nurses, medical assistants, advanced practice providers, physicians, social workers) a one-time survey (N = 29) and interview (N = 25). Quantitative outcomes were assessed using previously validated feasibility, acceptability, and appropriateness measures. Additional qualitative data were collected to provide context for interpreting the scores.

RESULTS

 Nurses and medical assistants, who were directly responsible for implementing SRS, reported low feasibility, acceptability, and appropriateness, compared with other team members (e.g., physicians, social workers, advanced practice providers). Team members identified potential improvements needed to optimize SRS, such as hiring additional staff, improving staff training, providing different modalities for screening completion among individuals with disabilities, and revising the patient-reported outcomes to improve suicide risk prediction.

CONCLUSION

 Staff perspectives about implementing SRS as a part of routine cancer care for HNC patients varied widely. Before screening can be implemented on a larger scale for HNC and other cancer patients, additional implementation strategies may be needed that optimize workflow and reduce staff burden, such as staff training, multiple modalities for completion, and refined tools for identifying which patients are at greatest risk for suicide.

Emerging Risk of Adolescent Depression and Suicide Detected Through Pediatric Primary Care Screening

OBJECTIVE

The goal of the current study was to document patterns of stability and change in adolescent depression and suicide risk detected via universal depression screening in pediatric primary care and to determine who may go on to experience emerging risk.

METHODS

Retrospective electronic health record information (sociodemographic data and depression screening results for 2 timepoints) was extracted for adolescents aged 12-17 who attended well-visits between November 15, 2017, and February 1, 2020, in a large pediatric primary care network. A total of 27,335 adolescents with 2 completed depression screeners were included in the current study.

RESULTS

While most adolescents remained at low risk for depression and suicide across the 2 timepoints, others experienced emerging risk (i.e., low risk at time 1 but elevated risk at time 2), decreasing risk (i.e., high risk at time 1 but low risk at time 2) or stable high risk for depression or suicide. Odds of experiencing emerging depression and suicide risk were higher among adolescents who were female (compared to males), Black (compared to White), and had Medicaid insurance (compared to private insurance). Odds of experiencing emerging depression risk were also higher among older adolescents (compared to younger adolescents) as well as adolescents who identified as Hispanic/Latino (compared to non-Hispanic/Latino).

CONCLUSIONS

Findings can inform symptom monitoring and opportunities for prevention in primary care.

Suicide Prevention in Pediatric Health Care Settings

Given recent trends demonstrating increased suicide risk among youth, particularly those from minoritized populations, youth suicide is a major public health concern. Evidence-based practices for the identification and management of youth suicide risk have been developed, yet many challenges exist to implementing them routinely in health care settings. Suggestions for leveraging publicly available resources, gathering input from a range of stakeholders to inform implementation, and enhancing multidisciplinary collaboration are provided with the aim of offering tangible steps toward addressing the youth suicide crisis.

Demographic predictors of emergency service utilization patterns in youth at risk of suicide

OBJECTIVE

To explore demographic predictors of Emergency Department (ED) utilization among youth with a history of suicidality (i.e., ideation or behaviors).

METHODS

Electronic health records were extracted from 2017 to 2021 for 3094 8-22 year-old patients with a history of suicidality at an urban academic medical center ED in the Mid-Atlantic. Logistic regression analyses were used to assess for demographic predictors of ED utilization frequency, timing of subsequent visits, and reasons for subsequent visits over a 24-month follow-up period.

RESULTS

Black race (OR = 1.45, 95% CI = 1.11-1.92), Female sex (OR = 1.59, 95% CI = 1.26-2.03), and having Medicaid insurance (OR = 1.71, 95% CI = 1.37-2.14) were associated with increased utilization, while being under 18 was associated with lower utilization (<12: OR = 0.38, 95% CI = 0.26-0.56; 12-18: OR = 0.47, 95% CI = 0.35-0.63). These demographics were also associated with ED readmission within 90 days, while being under 18 was associated with a lower odds of readmission.

CONCLUSIONS

Among patients with a history of suicidality, those who identify as Black, young adults, patients with Medicaid, and female patients were more likely to be frequent utilizers of the ED within the 2 years following their initial visit. This pattern may suggest inadequate health care access for these groups, and a need to develop better care coordination with an intersectional focus to facilitate utilization of other health services.

Screening and Assessing Suicide Risk in Medical Settings: Feasible Strategies for Early Detection

Early detection of risk is a key suicide prevention strategy. Given that most individuals who die by suicide visit a health care provider in the year leading up to their death, medical settings are ideal venues for identifying those at elevated risk and bridging them to life-saving care. Clinicians are presented with an opportunity to engage in proactive suicide prevention efforts through practical and adaptable suicide risk screening, assessment, and management processes. Psychiatrists and mental health clinicians are well positioned to assist nonpsychiatric clinicians on the frontlines of this public health problem. This article discusses the importance of identifying people at elevated suicide risk through screening, differentiates screening from assessment procedures, and presents practical strategies for implementing evidence-based screening and assessment tools into practice as part of a three-tiered clinical pathway. Specifically, this article discusses key components that guide embedding suicide prevention strategies into the workflows of busy medical settings.

Tools to Detect Risk of Death by Suicide: A Systematic Review and Meta-Analysis

Objective: There is limited knowledge about the ability of instruments to detect risk of suicide in a range of settings. Prior reviews have not considered whether the utility of instruments depends on prior probability of risk. We performed a systematic review to determine the diagnostic accuracy of instruments to detect risk of suicide in adults using likelihood ratio analysis. This method aids evaluation of prior probabilities of risk.

Data Sources: We searched MEDLINE, Cochrane Database of Systematic Reviews, PsycINFO, EMBASE, and Scopus from inception through January 19, 2021.

Study Selection: We included clinical trials, observational studies, and quasi-experimental studies assessing the diagnostic accuracy of instruments to detect risk of suicide in adults. There were no language restrictions.

Data Extraction: Three reviewers in duplicate assessed full texts to determine eligibility and extracted data from included studies. Positive (LR+) and negative likelihood ratio (LR-) and 95% CIs were calculated for each instrument.

Results: Thirty studies met inclusion criteria. Most instruments showed minimal utility to detect or rule out risk of suicide, with LR+ ≤ 2.0 and LR- ≥ 0.5. A few instruments had a high utility for improving risk detection in emergency department, inpatient mental health, and prison settings when patients scored above the cutoff (LR+ > 10). For example, among patients discharged from an emergency department, the Columbia Suicide Severity Rating Scale-Clinical Practice Screener had a LR+ of 10.3 (95% CI, 6.3-16.8) at 3-month follow-up. The clinical utility of the instruments depends on the pretest probability of suicide in the setting. Because studies spanned over 6 decades, the findings are at risk for secular trends.

Discussion: We identified several instruments that may hold promise for detecting risk of suicide in emergency department, inpatient mental health, or prison settings. The utility of the instrument hinges, in part, on baseline suicide risk.

Registration: PROSPERO: CRD42021285528.

Suicide Screening Tools for Pediatric Emergency Department Patients: A Systematic Review

BACKGROUND

According to the Centers for Disease Control and Prevention, suicidality and suicidal behavior among youth continues to increase significantly each year. Many of those who die by suicide interact with health services in the year before death. This systematic review sought to identify and describe empirically tested screening tools for suicidality in youth presenting to Emergency Departments (ED).

OBJECTIVE

(1) To identify and compare existing tools used to screen for suicidality in children and adolescents who present to the ED and (2) to ascertain the prevalence of suicidality in pediatric populations found with these tools.

METHODS

We searched Ovid Medline, CINAHL, Scopus, and Cochrane databases for primary research studies that identified and evaluated screening tools for suicide risk in pediatric ED patients. A total of 7,597 publications published before August 25, 2021 met search criteria and were screened by two independent reviewers based on our inclusion and exclusion criteria, with any conflicts resolved via consensus meetings or an independent reviewer. A total of 110 papers were selected for full text review, of which 67 were excluded upon further inspection. Covidence was used to extract and synthesize results.

RESULTS

43 articles were eligible for inclusion. Most studies (n = 33) took place in general pediatric EDs; the quality was generally high. Patients ranged from 4-24 years old, with most screening tested in patients 12 years and older. The most researched tools were the Ask-Suicide Screening Questions (ASQ) (n = 15), Columbia-Suicide Severity Rating Scale (C-SSRS) (n = 12), Suicidal Ideation Questionnaire (SIQ) (n = 11), and the Risk of Suicide Questionnaire (RSQ) (n = 7). Where screening was applied to all patients, about one-fifth of pediatric ED patients screened positive; where suicide screening was applied to psychiatric patients only, over half screened positive. Positive screens were more likely to be female and older than negative screens and they were more likely to be assessed and admitted.

CONCLUSION

Several validated screening tools exist for the purpose of screening pediatric populations in EDs for suicidality. Such tools may help to support early detection and appropriate intervention for youth at risk of suicide.

SYSTEMATIC REVIEW REGISTRATION

https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=276328, identifier: 276328.