Predicting Outcomes of Antidepressant Treatment in Community Practice Settings

OBJECTIVE

The authors examined whether machine-learning models could be used to analyze data from electronic health records (EHRs) to predict patients' responses to antidepressant medications.

METHODS

EHR data from a Washington State health system identified patients ages ≥13 years who started an antidepressant medication in 2016 in a community practice setting and had a baseline Patient Health Questionnaire-9 (PHQ-9) score of ≥10 and at least one PHQ-9 score recorded 14-180 days later. Potential predictors of a response to antidepressants were extracted from the EHR and included demographic characteristics, psychiatric and substance use diagnoses, past psychiatric medication use, mental health service use, and past PHQ-9 scores. Random-forest and penalized regression analyses were used to build models predicting follow-up PHQ-9 score and a favorable treatment response (≥50% improvement in score).

RESULTS

Among 2,469 patients starting antidepressant medication treatment, the mean±SD baseline PHQ-9 score was 17.3±4.5, and the mean lowest follow-up score was 9.2±5.9. Outcome data were available for 72% of the patients. About 48% of the patients had a favorable treatment response. The best-fitting random-forest models yielded a correlation between predicted and observed follow-up scores of 0.38 (95% CI=0.32-0.45) and an area under the receiver operating characteristic curve for a favorable response of 0.57 (95% CI=0.52-0.61). Results were similar for penalized regression models and for models predicting last PHQ-9 score during follow-up.

CONCLUSIONS

Prediction models using EHR data were not accurate enough to inform recommendations for or against starting antidepressant medication. Personalization of depression treatment should instead rely on systematic assessment of early outcomes.

Adolescents Who Do Not Endorse Risk via the Patient Health Questionnaire Before Self-Harm or Suicide

Importance

Given that the Patient Health Questionnaire (PHQ) item 9 is commonly used to screen for risk of self-harm and suicide, it is important that clinicians recognize circumstances when at-risk adolescents may go undetected.

Objective

To understand characteristics of adolescents with a history of depression who do not endorse the PHQ item 9 before a near-term intentional self-harm event or suicide.

Design, Setting, and Participants

This was a retrospective cohort study design using electronic health record and claims data from January 2009 through September 2017. Settings included primary care and mental health specialty clinics across 7 integrated US health care systems. Included in the study were adolescents aged 13 to 17 years with history of depression who completed the PHQ item 9 within 30 or 90 days before self-harm or suicide. Study data were analyzed September 2022 to April 2023.

Exposures

Demographic, diagnostic, treatment, and health care utilization characteristics.

Main Outcome(s) and Measure(s)

Responded "not at all" (score = 0) to PHQ item 9 regarding thoughts of death or self-harm within 30 or 90 days before self-harm or suicide.

Results

The study included 691 adolescents (mean [SD] age, 15.3 [1.3] years; 541 female [78.3%]) in the 30-day cohort and 1024 adolescents (mean [SD] age, 15.3 [1.3] years; 791 female [77.2%]) in the 90-day cohort. A total of 197 of 691 adolescents (29%) and 330 of 1024 adolescents (32%), respectively, scored 0 before self-harm or suicide on the PHQ item 9 in the 30- and 90-day cohorts. Adolescents seen in primary care (odds ratio [OR], 1.5; 95% CI, 1.0-2.1; P = .03) and older adolescents (OR, 1.2; 95% CI, 1.0-1.3; P = .02) had increased odds of scoring 0 within 90 days of a self-harm event or suicide, and adolescents with a history of inpatient hospitalization and a mental health diagnosis had twice the odds (OR, 2.0; 95% CI, 1.3-3.0; P = .001) of scoring 0 within 30 days. Conversely, adolescents with diagnoses of eating disorders were significantly less likely to score 0 on item 9 (OR, 0.4; 95% CI, 0.2-0.8; P = .007) within 90 days.

Conclusions and Relevance

Study results suggest that older age, history of an inpatient mental health encounter, or being screened in primary care were associated with at-risk adolescents being less likely to endorse having thoughts of death and self-harm on the PHQ item 9 before a self-harm event or suicide death. As use of the PHQ becomes more widespread in practice, additional research is needed for understanding reasons why many at-risk adolescents do not endorse thoughts of death and self-harm.

Reducing Firearm Access for Suicide Prevention: Implementation Evaluation of the Web-Based "Lock to Live" Decision Aid in Routine Health Care Encounters

Background

"Lock to Live" (L2L) is a novel web-based decision aid for helping people at risk of suicide reduce access to firearms. Researchers have demonstrated that L2L is feasible to use and acceptable to patients, but little is known about how to implement L2L during web-based mental health care and in-person contact with clinicians.

Objective

The goal of this project was to support the implementation and evaluation of L2L during routine primary care and mental health specialty web-based and in-person encounters.

Methods

The L2L implementation and evaluation took place at Kaiser Permanente Washington (KPWA)-a large, regional, nonprofit health care system. Three dimensions from the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) model-Reach, Adoption, and Implementation-were selected to inform and evaluate the implementation of L2L at KPWA (January 1, 2020, to December 31, 2021). Electronic health record (EHR) data were used to purposefully recruit adult patients, including firearm owners and patients reporting suicidality, to participate in semistructured interviews. Interview themes were used to facilitate L2L implementation and inform subsequent semistructured interviews with clinicians responsible for suicide risk mitigation. Audio-recorded interviews were conducted via the web, transcribed, and coded, using a rapid qualitative inquiry approach. A descriptive analysis of EHR data was performed to summarize L2L reach and adoption among patients identified at high risk of suicide.

Results

The initial implementation consisted of updates for clinicians to add a URL and QR code referencing L2L to the safety planning EHR templates. Recommendations about introducing L2L were subsequently derived from the thematic analysis of semistructured interviews with patients (n=36), which included (1) "have an open conversation," (2) "validate their situation," (3) "share what to expect," (4) "make it accessible and memorable," and (5) "walk through the tool." Clinicians' interviews (n=30) showed a strong preference to have L2L included by default in the EHR-based safety planning template (in contrast to adding it manually). During the 2-year observation period, 2739 patients reported prior-month suicide attempt planning or intent and had a documented safety plan during the study period, including 745 (27.2%) who also received L2L. Over four 6-month subperiods of the observation period, L2L adoption rates increased substantially from 2% to 29% among primary care clinicians and from <1% to 48% among mental health clinicians.

Conclusions

Understanding the value of L2L from users' perspectives was essential for facilitating implementation and increasing patient reach and clinician adoption. Incorporating L2L into the existing system-level, EHR-based safety plan template reduced the effort to use L2L and was likely the most impactful implementation strategy. As rising suicide rates galvanize the urgency of prevention, the findings from this project, including L2L implementation tools and strategies, will support efforts to promote safety for suicide prevention in health care nationwide.

Predicting suicide death after emergency department visits with mental health or self-harm diagnoses

OBJECTIVE

Use health records data to predict suicide death following emergency department visits.

METHODS

Electronic health records and insurance claims from seven health systems were used to: identify emergency department visits with mental health or self-harm diagnoses by members aged 11 or older; extract approximately 2500 potential predictors including demographic, historical, and baseline clinical characteristics; and ascertain subsequent deaths by self-harm. Logistic regression with lasso and random forest models predicted self-harm death over 90 days after each visit.

RESULTS

Records identified 2,069,170 eligible visits, 899 followed by suicide death within 90 days. The best-fitting logistic regression with lasso model yielded an area under the receiver operating curve of 0.823 (95% CI 0.810-0.836). Visits above the 95th percentile of predicted risk included 34.8% (95% CI 31.1-38.7) of subsequent suicide deaths and had a 0.303% (95% CI 0.261-0.346) suicide death rate over the following 90 days. Model performance was similar across subgroups defined by age, sex, race, and ethnicity.

CONCLUSIONS

Machine learning models using coded data from health records have moderate performance in predicting suicide death following emergency department visits for mental health or self-harm diagnosis and could be used to identify patients needing more systematic follow-up.

Transitions in health insurance among continuously insured patients with schizophrenia

Changes in health insurance coverage may disrupt access to and continuity of care, even for those who remain insured. Continuity of care is especially important in schizophrenia, which requires ongoing medical and pharmaceutical treatment. However, little is known about continuity of insurance coverage among those with schizophrenia. The objective was to examine the probability of insurance transitions for individuals with schizophrenia who were continuously insured and whether this varied across insurance types. The Massachusetts All-Payer Claims Database identified individuals with schizophrenia aged 18-64 who were continuously insured during a two-year period between 2014 and 2018. A logistic regression estimated the association of having an insurance transition - defined as having a change in insurance type - with insurance type at the start of the period, adjusting for age, sex, ZIP code in the lowest quartile of median income, and ZIP code with concentrated poverty. Overall, 15.1% had at least one insurance transition across a 24-month period. Insurance transitions were most frequent among those with plans from the Marketplace. In regression adjusted results, individuals covered by the traditional Medicaid program were 20.2 percentage points [pp] (95% confidence interval [CI]: 24.6 pp, 15.9 pp) less likely to have an insurance transition than those who were insured by a Marketplace plan. Insurance transitions among individuals with schizophrenia were common, with more than one in six people having at least one transition in insurance type during a two-year period. Given that even continuously insured individuals with schizophrenia commonly experience insurance transitions, attention to insurance transitions as a barrier to care access and continuity is warranted.

Psychotherapy Disruption Before and After the Transition to Virtual Mental Health Care Induced by the COVID-19 Pandemic

OBJECTIVE

This study aimed to examine population-level disruption in psychotherapy before and after the rapid shift to virtual mental health care induced by the onset of the COVID-19 pandemic in the United States.

METHODS

This retrospective study used electronic health record and insurance claims data from three U.S. health systems. The sample included 110,089 patients with mental health conditions who were members of the health systems' affiliated health plans and attended at least two psychotherapy visits from June 14, 2019, through December 15, 2020. Data were subdivided into two 9-month periods (before vs. after COVID-19 onset, defined in this study as March 14, 2020). Psychotherapy visits were measured via health records and categorized as in person or virtual. Disruption was defined as a gap of >45 days between visits.

RESULTS

Visits in the preonset period were almost exclusively in person (97%), whereas over half of visits in the postonset period were virtual (52%). Approximately 35% of psychotherapy visits were followed by a disruption in the preonset period, compared with 18% in the postonset period. Disruption continued to be less common (adjusted OR=0.45) during the postonset period after adjustment for visit, mental health, and sociodemographic factors. The magnitude of the difference in disruption between periods was homogeneous across sociodemographic characteristics but heterogeneous across psychiatric diagnoses.

CONCLUSIONS

This study found fewer population-level disruptions in psychotherapy receipt after rapid transition to virtual mental health care following COVID-19 onset. These data support the continued availability of virtual psychotherapy.

Stability of Suicide Risk Prediction Models During Changes in Health Care Delivery

OBJECTIVE

The authors aimed to use health records data to examine how the accuracy of statistical models predicting self-harm or suicide changed between 2015 and 2019, as health systems implemented suicide prevention programs.

METHODS

Data from four large health systems were used to identify specialty mental health visits by patients ages ≥11 years, assess 311 potential predictors of self-harm (including demographic characteristics, historical risk factors, and index visit characteristics), and ascertain fatal or nonfatal self-harm events over 90 days after each visit. New prediction models were developed with logistic regression with LASSO (least absolute shrinkage and selection operator) in random samples of visits (65%) from each calendar year and were validated in the remaining portion of the sample (35%).

RESULTS

A model developed for visits from 2009 to mid-2015 showed similar classification performance and calibration accuracy in a new sample of about 13.1 million visits from late 2015 to 2019. Area under the receiver operating characteristic curve (AUC) ranged from 0.840 to 0.849 in the new sample, compared with 0.851 in the original sample. New models developed for each year for 2015-2019 had classification performance (AUC range 0.790-0.853), sensitivity, and positive predictive value similar to those of the previously developed model. Models selected similar predictors from 2015 to 2019, except for more frequent selection of depression questionnaire data in later years, when questionnaires were more frequently recorded.

CONCLUSIONS

A self-harm prediction model developed with 2009-2015 visit data performed similarly when applied to 2015-2019 visits. New models did not yield superior performance or identify different predictors.

Racial-Ethnic Differences in Receipt of Past-Year Health Care Services Among Suicide Decedents: A Case-Control Study

OBJECTIVE

Suicide remains an urgent public health crisis. Although some sociodemographic characteristics are associated with greater suicide risk in the general population, it is unclear whether individuals utilizing health care in the United States have similar suicide incidence patterns. The authors examined whether race-ethnicity is associated with suicide death among patients seeking health care and investigated health care utilization patterns.

METHODS

Data were collected from electronic health records and government mortality records for patients seeking health care across nine health care systems in the United States. Patients who died by suicide (N=1,935) were matched with patients in a control group (N=19,350) within each health care system.

RESULTS

Patients who died by suicide were significantly more likely to be White, older, male, living in low-education areas, living in rural areas, or diagnosed as having mental health conditions or were significantly less likely to have commercial insurance (p<0.05). Among most racial-ethnic groups, those who died by suicide had a higher number of past-year mental health, primary care, and total health care visits; for American Indian/Alaska Native patients, the number of health care visits tended to be lower among suicide decedents.

CONCLUSIONS

These findings suggest that higher past-year health care utilization was associated with increased likelihood of suicide death across several racial-ethnic groups. This observation underscores the need for identifying and managing suicide risk in health care settings, including outside of mental health visits, among most racial-ethnic groups.

Predicting risk of suicidal behavior from insurance claims data vs. linked data from insurance claims and electronic health records

PURPOSE

Observational studies assessing effects of medical products on suicidal behavior often rely on health record data to account for pre-existing risk. We assess whether high-dimensional models predicting suicide risk using data derived from insurance claims and electronic health records (EHRs) are superior to models using data from insurance claims alone.

METHODS

Data were from seven large health systems identified outpatient mental health visits by patients aged 11 or older between 1/1/2009 and 9/30/2017. Data for the 5 years prior to each visit identified potential predictors of suicidal behavior typically available from insurance claims (e.g., mental health diagnoses, procedure codes, medication dispensings) and additional potential predictors available from EHRs (self-reported race and ethnicity, responses to Patient Health Questionnaire or PHQ-9 depression questionnaires). Nonfatal self-harm events following each visit were identified from insurance claims data and fatal self-harm events were identified by linkage to state mortality records. Random forest models predicting nonfatal or fatal self-harm over 90 days following each visit were developed in a 70% random sample of visits and validated in a held-out sample of 30%. Performance of models using linked claims and EHR data was compared to models using claims data only.

RESULTS

Among 15 845 047 encounters by 1 574 612 patients, 99 098 (0.6%) were followed by a self-harm event within 90 days. Overall classification performance did not differ between the best-fitting model using all data (area under the receiver operating curve or AUC = 0.846, 95% CI 0.839-0.854) and the best-fitting model limited to data available from insurance claims (AUC = 0.846, 95% CI 0.838-0.853). Competing models showed similar classification performance across a range of cut-points and similar calibration performance across a range of risk strata. Results were similar when the sample was limited to health systems and time periods where PHQ-9 depression questionnaires were recorded more frequently.

CONCLUSION

Investigators using health record data to account for pre-existing risk in observational studies of suicidal behavior need not limit that research to databases including linked EHR data.

The rapid shift to virtual mental health care: Examining psychotherapy disruption by rurality status

BACKGROUND

Given the low usage of virtual health care prior to the COVID-19 pandemic, it was unclear whether those living in rural locations would benefit from increased availability of virtual mental health care. The rapid transition to virtual services during the COVID-19 pandemic allowed for a unique opportunity to examine how the transition to virtual mental health care impacted psychotherapy disruption (i.e., 45+ days between appointments) among individuals living in rural locations compared with those living in nonrural locations.

METHODS

Electronic health record and insurance claims data were collected from three health care systems in the United States including rurality status and psychotherapy disruption. Psychotherapy disruption was measured before and after the COVID-19 pandemic onset.

RESULTS

Both the nonrural and rural cohorts had significant decreases in the rates of psychotherapy disruption from pre- to post-COVID-19 onset (32.5-16.0% and 44.7-24.8%, respectively, p < 0.001). The nonrural cohort had a greater reduction of in-person visits compared with the rural cohort (96.6-45.0 vs. 98.0-66.2%, respectively, p < 0.001). Among the rural cohort, those who were younger and those with lower education had greater reductions in psychotherapy disruption rates from pre- to post-COVID-19 onset. Several mental health disorders were associated with experiencing psychotherapy disruption.

CONCLUSIONS

Though the rapid transition to virtual mental health care decreased the rate of psychotherapy disruption for those living in rural locations, the reduction was less compared with nonrural locations. Other strategies are needed to improve psychotherapy disruption, especially among rural locations (i.e., telephone visits).

Association Between Adverse Social Determinants of Health and Suicide Death

OBJECTIVE

The aim of this study was to identify adverse social determinants of health (SDoH) International Statistical Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code prevalence among individuals who died by suicide and to examine associations between documented adverse SDoH and suicide.

RESEARCH DESIGN

A case-control study using linked medical record, insurance claim, and mortality data from 2000 to 2015 obtained from 9 Mental Health Research Network-affiliated health systems. We included 3330 individuals who died by suicide and 333,000 randomly selected controls matched on index year and health system location. All individuals in the study (cases and controls) had at least 10 months of enrollment before the study index date. The index date for the study for each case and their matched controls was the suicide date for that given case.

RESULTS

Adverse SDoH documentation was low; only 6.6% of cases had ≥1 documented adverse SDoH in the year before suicide. Any documented SDoH and several specific adverse SDoH categories were more frequent among cases than controls. Any documented adverse SDoH was associated with higher suicide odds [adjusted odds ratio (aOR)=2.76; 95% CI: 2.38-3.20], as was family alcoholism/drug addiction (aOR=18.23; 95% CI: 8.54-38.92), being an abuse victim/perpetrator (aOR=2.53; 95% CI: 1.99-3.21), other primary support group problems (aOR=1.91; 95% CI: 1.32-2.75), employment/occupational maladjustment problems (aOR=8.83; 95% CI: 5.62-13.87), housing/economic problems (aOR: 6.41; 95% CI: 4.47-9.19), legal problems (aOR=27.30; 95% CI: 12.35-60.33), and other psychosocial problems (aOR=2.58; 95% CI: 1.98-3.36).

CONCLUSIONS

Although documented SDoH prevalence was low, several adverse SDoH were associated with increased suicide odds, supporting calls to increase SDoH documentation in medical records. This will improve understanding of SDoH prevalence and assist in identification and intervention among individuals at high suicide risk.

Beyond clinical outcomes: Case control study of the role of race in disruptive life events for people with serious mental illness

OBJECTIVE

To understand how race and serious mental illness (SMI) interact for disruptive life events defined as financial (bankruptcy and judgement filings), and non-financial (arrests).

METHODS

Patients were adults with schizophrenia (SCZ; N = 16,159) or bipolar I disorder (BPI; N = 30,008) matched 1:1 to patients without SMI (non-SMI) from health systems in Michigan and Southern California during 1/1/2007 through 12/31/2018. The main exposure was self-reported race, and the outcome was disruptive life events aggregated by Transunion. We hypothesized that Black patients with SCZ or BPI would be the most likely to experience a disruptive life event when compared to Black patients without SMI, and all White or Asian patients regardless of mental illness.

RESULTS

Black patients with SCZ had the least likelihood (37% lower) and Asian patients with BPI had the greatest likelihood (2.25 times higher) of experiencing a financial disruptive life event among all patients in the study. There was no interaction of race with either SCZ or BPI for experiencing an arrest. The findings did not support our hypotheses for patients with SCZ and partially supported them for patients with BPI.

CONCLUSIONS

Clinical initiatives to assess social determinants of health should consider a focus on Asian patients with BPI.

Assessing the differential item functioning of PHQ-9 items for diverse racial and ethnic adults with mental health and/or substance use disorder diagnoses: A retrospective cohort study

BACKGROUND

Improving health equity in depression care and suicide screening requires that measures like the Patient Health Questionnaire 9 (PHQ-9) function similarly for diverse racial and ethnic groups. We evaluated PHQ-9 differential item functioning (DIF) between racial/ethnic groups in a retrospective cohort study of secondary electronic health record (EHR) data from eight healthcare systems.

METHODS

The population (n = 755,156) included patients aged 18-64 with mental health and/or substance use disorder (SUD) diagnoses who had a PHQ-9 with no missing item data in the EHR for primary care or mental health visits between 1/1/2009-9/30/2017. We drew two random samples of 1000 from the following racial/ethnic groups originally recorded in EHRs (n = 14,000): Hispanic, and non-Hispanic White, Black, Asian, American Indian/Alaska Native, Native Hawaiian/Other Pacific Islander, multiracial. We assessed DIF using iterative hybrid ordinal logistic regression and item response theory with p < 0.01 and 1000 Monte Carlo simulations, where change in model R2 > 0.01 represented non-negligible (e.g., clinically meaningful) DIF.

RESULTS

All PHQ-9 items displayed statistically significant, but negligible (e.g., clinically unmeaningful) DIF between compared groups. The negligible DIF varied between random samples, although six items showed negligible DIF between the same comparison groups in both random samples.

LIMITATIONS

Our findings may not generalize to disaggregated racial/ethnic groups or persons without mental health and/or SUD diagnoses.

CONCLUSIONS

We found the PHQ-9 had clinically unmeaningful cross-cultural DIF for adult patients with mental health and/or SUD diagnoses. Future research could disaggregate race/ethnicity to discern if within-group identification impacts PHQ-9 DIF.

Diagnosed Posttraumatic Stress Disorder and Other Trauma-Associated Stress Disorders and Risk for Suicide Mortality

OBJECTIVE

Strong evidence exists for posttraumatic stress disorder (PTSD) as a risk factor for suicidal thoughts and behaviors across diverse populations. However, few empirical studies have examined PTSD and other trauma-associated stress disorders as risk factors for suicide mortality among health system populations. This study aimed to assess trauma-associated stress diagnoses as risk factors for suicide mortality in a U.S. health system population.

METHODS

This case-control, matched-design study examined individuals who died by suicide between 2000 and 2015 and had received care from nine U.S. health systems affiliated with the Mental Health Research Network (N=3,330). Individuals who died by suicide were matched with individuals from the general health system population (N=333,000): 120 individuals with PTSD who died by suicide were matched with 1,592 control group members, 84 with acute reaction to stress were matched with 2,218 control individuals, and 331 with other stress reactions were matched with 8,174 control individuals.

RESULTS

After analyses were adjusted for age and sex, individuals with any trauma-associated stress condition were more likely to have died by suicide. Risk was highest among individuals with PTSD (adjusted OR [AOR]=10.10, 95% CI=8.31-12.27), followed by those with other stress reactions (AOR=5.38, 95% CI=4.78-6.06) and those with acute reaction to stress (AOR=4.49, 95% CI=3.58-5.62). Patterns of risk remained the same when the analyses were adjusted for any comorbid psychiatric condition.

CONCLUSIONS

All trauma-associated stress disorders are risk factors for suicide mortality, highlighting the importance of health system suicide prevention protocols that consider the full spectrum of traumatic stress diagnoses.

Factors associated with suicide mortality among reproductive age women: a case-control study

OBJECTIVE

Examine demographic, psychosocial, pregnancy-related, and healthcare utilisation factors associated with suicide mortality among reproductive age women.

METHODS

Data from nine health care systems in the Mental Health Research Network were included. A case-control study design was used in which 290 reproductive age women who died by suicide (cases) from 2000 to 2015 were matched with 2,900 reproductive age women from the same healthcare system who did not die by suicide (controls). Conditional logistic regression was used to analyse associations between patient characteristics and suicide.

RESULTS

Women of reproductive age who died by suicide were more likely to have mental health (aOR = 7.08, 95% CI: 5.17, 9.71) or substance use disorders (aOR = 3.16, 95% CI: 2.19, 4.56) and to have visited the emergency department in the year prior to index date (aOR = 3.47, 95% CI: 2.50, 4.80). Non-Hispanic White women (aOR = 0.70, 95% CI: 0.51, 0.97) and perinatal (pregnant or postpartum) women were less likely to have died by suicide (aOR = 0.27, 95% CI: 0.13, 0.58).

CONCLUSIONS

Reproductive age women with mental health and/or substance use disorders, prior emergency department encounters, or who are of racial or ethnic minority status were at increased risk of suicide mortality and may benefit from routine screening and monitoring. Future research should further examine the relationship between pregnancy-related factors and suicide mortality.

Assessment of Disruptive Life Events for Individuals Diagnosed With Schizophrenia or Bipolar I Disorder Using Data From a Consumer Credit Reporting Agency

Importance

There is a dearth of population-level data on major disruptive life events (defined here as arrests by a legal authority, address changes, bankruptcy, lien, and judgment filings) for patients with bipolar I disorder (BPI) or schizophrenia, which has limited studies on mental health and treatment outcomes.

Objective

To conduct a population-level study on disruptive life events by using publicly available data on disruptive life events, aggregated by a consumer credit reporting agency in conjunction with electronic health record (EHR) data.

Design, Setting, and Participants

This study used EHR data from 2 large, integrated health care systems, Kaiser Permanente Southern California and Henry Ford Health. Cohorts of patients diagnosed from 2007 to 2019 with BPI or schizophrenia were matched 1:1 by age at analysis, age at diagnosis (if applicable), sex, race and ethnicity, and Medicaid status to (1) an active comparison group with diagnoses of major depressive disorder (MDD) and (2) a general health (GH) cohort without diagnoses of BPI, schizophrenia, or MDD. Patients with diagnoses of BPI or schizophrenia and their respective comparison cohorts were matched to public records data aggregated by a consumer credit reporting agency (98% match rate). Analysis took place between November 2020 and December 2022.

Main Outcomes and Measures

The differences in the occurrence of disruptive life events among patients with BPI or schizophrenia and their comparison groups.

Results

Of 46 167 patients, 30 008 (65%) had BPI (mean [SD] age, 42.6 [14.2] years) and 16 159 (35%) had schizophrenia (mean [SD], 41.4 [15.1] years). The majoriy of patients were White (30 167 [65%]). In addition, 18 500 patients with BPI (62%) and 6552 patients with schizophrenia (41%) were female. Patients with BPI were more likely to change addresses than patients in either comparison cohort (with the incidence ratio being as high as 1.25 [95% CI, 1.23-1.28]) when compared with GH cohort. Patients with BPI were also more likely to experience any of the financial disruptive life events with odds ratio ranging from 1.15 [95% CI, 1.07-1.24] to 1.50 [95% CI, 1.42-1.58]). The largest differences in disruptive life events were seen in arrests of patients with either BPI or schizophrenia compared with GH peers (3.27 [95% CI, 2.84-3.78] and 3.04 [95% CI, 2.57-3.59], respectively). Patients with schizophrenia had fewer address changes and were less likely to experience a financial event than their matched comparison cohorts.

Conclusions and Relevance

This study demonstrated that data aggregated by a consumer credit reporting agency can support population-level studies on disruptive life events among patients with BPI or schizophrenia.

Depression Treatment Initiation Among Patients With Versus Without Chronic Pain

OBJECTIVE

The purpose of this study was to examine the extent to which the presence of chronic noncancer pain (CNCP) impacts the likelihood that patients with diagnoses of depression will initiate depression treatment compared with those without CNCP.

METHODS

We performed a retrospective cohort study of Kaiser Permanente of Georgia members older than 18 years who received a diagnosis of depression. Demographics and medical history were extracted from the electronic health record database. Members were further classified by the presence or absence of a CNCP diagnosis. Outcomes of interest were treated as time dependent and included ( 1 ) time to fulfillment of a new antidepressant medication and ( 2 ) time to a follow-up mental health encounter. Outcomes were compared between members with and without a CNCP diagnosis using Kaplan-Meier survival curves and Cox proportional hazard regression models.

RESULTS

During the study period, 22,996 members met the inclusion criteria and 27.4% had a diagnosis of CNCP. In the matched sample, there was no difference in the time to a new antidepressant fill among members with and without CNCP (hazard ratio = 0.96; 95% confidence interval = 0.90-1.02; p = .18). In contrast, members with CNCP were significantly less likely to have a new mental health encounter after diagnosis (hazard ratio = 0.87; 95% confidence interval = 0.81-0.94; p < .001).

CONCLUSIONS

Patients with CNCP were significantly less likely to have a new mental health encounter after a depression diagnosis compared with patients without CNCP. Additional outreach and consideration may be needed to improve initiation of depression treatment for newly diagnosed patients with comorbid depression and CNCP.

Understanding TBI as a Risk Factor Versus a Means of Suicide Death Using Electronic Health Record Data

OBJECTIVE

The aim of this research was to examine predictors and characterize causes of suicide death in people with traumatic brain injury (TBI) and conduct sensitivity analyses with and without people whose first diagnosis of TBI occurred within 3 days of their suicide death.

METHODS

This case-control study examined suicide risk for people with TBI in eight Mental Health Research Network-affiliated healthcare systems. Sample 1 included 61 persons with TBI who died by suicide and their 75 matched controls with TBI who did not die by suicide between January 1, 2000, and December 31, 2013. Sample 2 excluded the 34 persons with TBI whose first TBI diagnosis occurred within 3 days of their suicide death and their 46 matched controls. Descriptive statistics characterized the sample stratified by cases and controls, while conditional logistic regression models estimated the adjusted odds of suicide.

RESULTS

Over half of suicide deaths occurred within 3 days of a person's first diagnosis of TBI in the larger sample. After excluding these persons, people with TBI were 2.84 (95% confidence interval [CI]: 2.15-2.73) times more likely to die by suicide than were people without TBI. Among those with TBI, men were 16.39 times (95% CI: 1.89-142.15) more likely to die by suicide than were women.

CONCLUSIONS

Accounting for TBI as a potential consequence of suicide attenuates the association between TBI and suicide, but a robust association persists-especially among men. Ultimately, all people with TBI should be carefully screened and monitored for suicide risk.HIGHLIGHTSPeople with traumatic brain injury (TBI) were at considerably elevated risk for suicide deathMen with TBI had significantly increased risk of suicide death compared to women with TBITBI timing suggests confusion of risk factors for and consequences of suicide.

Differences in Insurance Coverage for Individuals With Schizophrenia After Implementation of the Patient Protection and Affordable Care Act

This cross-sectional study uses Medical Expenditure Panel Survey data to assess differences in insurance coverage for individuals with schizophrenia before vs after implementation of the Patient Protection and Affordable Care Act (ACA).

Fulfilling the Goals of 988 Through Crisis Stabilization Care

Recent implementation of the nationwide 988 Suicide and Crisis Lifeline has expanded telephone-based mental health crisis services and created a unified framework for crisis care in the United States. However, the infrastructure for the final step of the crisis continuum-an appropriate mental health service for persons in crisis to receive the care they need-is fragmented, unevenly distributed, underfunded, and understudied. Given the few options for individuals in crisis, most often inpatient psychiatric hospitals are the default option. In this Open Forum, the authors describe the scope of the problem and propose how clinicians, policy makers, and researchers can improve the availability of evidence-based disposition options for individuals in crisis.

Implementation of Suicide Risk Estimation Analytics to Support Mental Health Care for Quality Improvement

This quality improvement study describes use of estimation analytics to augment existing suicide prevention practices during routine mental health specialty encounters at a large US health care system.

Patient and Clinician Perspectives of a Standardized Question About Firearm Access to Support Suicide Prevention: A Qualitative Study

Importance

US residents report broad access to firearms, which are the most common means of suicide death in the US. Standardized firearm access questions during routine health care encounters are uncommon despite potential benefits for suicide prevention.

Objective

To explore patient and clinician experiences with a standard question about firearm access on a self-administered mental health questionnaire routinely used prior to primary care and mental health specialty encounters.

Design, Setting, and Participants

Qualitative semistructured interviews were conducted from November 18, 2019, to October 8, 2020, at Kaiser Permanente Washington, a large integrated care delivery system and insurance provider. Electronic health record data identified adult patients with a documented mental health diagnosis who had received a standard question about firearm access ("Do you have access to guns? yes/no") within the prior 2 weeks. A stratified sampling distribution selected 30% who answered "yes," 30% who answered "no," and 40% who left the question blank. Two groups of clinicians responsible for safety planning with patients at risk of suicide were also sampled: (1) licensed clinical social workers (LICSWs) in primary and urgent care settings and (2) consulting nurses (RNs).

Main Outcomes and Measures

Participants completed semistructured telephone interviews, which were recorded and transcribed. Directive (deductive) and conventional (inductive) content analyses were used to apply knowledge from prior research and describe new information. Thematic analysis was used to organize key content, and triangulation was used to describe the intersections between patient and clinician perspectives.

Results

Thirty-six patients were interviewed (of 76 sampled; mean [SD] age, 47.3 [17.9] years; 19 [53%] were male; 27 [75%] were White; 3 [8%] were Black; and 1 [3%] was Latinx or Hispanic. Sixteen participants had reported firearm access and 15 had reported thoughts of self-harm on the questionnaire used for sampling. Thirty clinicians were interviewed (of 51 sampled) (mean [SD] age, 44.3 [12.1] years; 24 [80%] were female; 18 [60%] were White; 5 [17%] were Asian or Pacific Islander; and 4 [13%] were Latinx or Hispanic) including 25 LICSWs and 5 RNs. Key organizing themes included perceived value of standardized questions about firearm access, challenges of asking and answering, and considerations for practice improvement. Clinician interview themes largely converged and/or complemented patient interviews.

Conclusions and Relevance

In this qualitative study using semistructured interviews with patients and clinicians, a standardized question about firearm access was found to encourage dialogue about firearm access. Respondents underscored the importance of nonjudgmental acknowledgment of patients' reasons for firearm access as key to patient-centered practice improvement.

Accuracy of ICD-10-CM encounter diagnoses from health records for identifying self-harm events

OBJECTIVE

Assess the accuracy of ICD-10-CM coding of self-harm injuries and poisonings to identify self-harm events.

MATERIALS AND METHODS

In 7 integrated health systems, records data identified patients reporting frequent suicidal ideation. Records then identified subsequent ICD-10-CM injury and poisoning codes indicating self-harm as well as selected codes in 3 categories where uncoded self-harm events might be found: injuries and poisonings coded as undetermined intent, those coded accidental, and injuries with no coding of intent. For injury and poisoning encounters with diagnoses in those 4 groups, relevant clinical text was extracted from records and assessed by a blinded panel regarding documentation of self-harm intent.

RESULTS

Diagnostic codes selected for review include all codes for self-harm, 43 codes for undetermined intent, 26 codes for accidental intent, and 46 codes for injuries without coding of intent. Clinical text was available for review for 285 events originally coded as self-harm, 85 coded as undetermined intent, 302 coded as accidents, and 438 injury events with no coding of intent. Blinded review of full-text clinical records found documentation of self-harm intent in 254 (89.1%) of those originally coded as self-harm, 24 (28.2%) of those coded as undetermined, 24 (7.9%) of those coded as accidental, and 48 (11.0%) of those without coding of intent.

CONCLUSIONS

Among patients at high risk, nearly 90% of injuries and poisonings with ICD-10-CM coding of self-harm have documentation of self-harm intent. Reliance on ICD-10-CM coding of intent to identify self-harm would fail to include a small proportion of true self-harm events.

Health Diagnoses and Service Utilization in the Year Before Youth and Young Adult Suicide

OBJECTIVE

Suicide rates among young people are rising. Health care visits provide opportunities for identification and intervention, yet studies have been limited by small or circumscribed samples. This study sought to expand the knowledge base by examining health care encounters and diagnoses among young people who later died by suicide.

METHODS

This case-control study examined diagnoses of mental and general medical disorders and health care utilization in the 30 and 365 days before suicide death in nine large U.S. health care systems. Data (years 2000-2015) from 445 suicide decedents ages 10-24 years were matched with data from 4,450 control group patients.

RESULTS

Suicide decedents were more likely to have at least one mental disorder diagnosis (51% vs. 16%; adjusted OR [AOR]=5.74, 95% CI=4.60-7.18) and had higher rates of nearly all mental health conditions. Substance use disorders were common (12%) and more likely (AOR=8.50, 95% CI=5.53-13.06) among suicide decedents. More than one in three (42%) suicide decedents had a health care visit in the month before death, and nearly all (88%) had a visit in the previous year.

CONCLUSIONS

Despite the greater likelihood of suicide associated with mental disorder diagnoses, such disorders were present among only 51% of suicide decedents. High rates of health care utilization among suicide decedents indicate a need for improving identification of mental health conditions and suicide risk across the health care system. Increased substance use screening may help identify youths at high risk because substance use disorders were significantly more prevalent and likely among suicide decedents.

Cancer and psychiatric diagnoses in the year preceding suicide

BACKGROUND

Patients with cancer are known to be at increased risk for suicide but little is known about the interaction between cancer and psychiatric diagnoses, another well-documented risk factor.

METHODS

Electronic medical records from nine healthcare systems participating in the Mental Health Research Network were aggregated to form a retrospective case-control study, with ICD-9 codes used to identify diagnoses in the 1 year prior to death by suicide for cases (N = 3330) or matching index date for controls (N = 297,034). Conditional logistic regression was used to assess differences in cancer and psychiatric diagnoses between cases and controls, controlling for sex and age.

RESULTS

Among patients without concurrent psychiatric diagnoses, cancer at disease sites with lower average 5-year survival rates were associated with significantly greater relative risk, while cancer disease sites with survival rates of >70% conferred no increased risk. Patients with most psychiatric diagnoses were at higher risk, however, there was no additional risk conferred to these patients by a concurrent cancer diagnosis.

CONCLUSION

We found no evidence of a synergistic effect between cancer and psychiatric diagnoses. However, cancer patients with a concurrent psychiatric illness remain at the highest relative risk for suicide, regardless of cancer disease site, due to strong independent associations between psychiatric diagnoses and suicide. For patients without a concurrent psychiatric illness, cancer disease sites associated with worse prognoses appeared to confer greater suicide risk.

Mother's and Father's Serious Mental Illness and Risk of Child Injury in a Taiwanese Birth Cohort

Objective: Parental serious mental illness (SMI) is associated with childhood injury. This study investigated whether child injury risk differs according to which parent is affected, SMI diagnosis, or timing of SMI onset. Methods: This cohort study included 1,999,322 singletons born in 2004-2014 identified from the national Taiwanese registries. General estimating equation Poisson models were used to estimate incidence rate ratios (IRRs) of injury events and hospitalizations before the age of 5 years among children according to which parent was affected, SMI diagnosis (schizophrenia [ICD-9-CM codes: 295, 297, 298.3, 298.4, 298.9], bipolar disorder [296.00-296.16, 296.40-296.81, 296.89-296.99, 298.1, 648.4], or major depressive disorder [MDD; 296.20-296.36, 296.82, 298.0]), and timing of diagnosis (before or after childbirth, as a proxy of timing of onset). Data analysis was performed on data obtained from April 20, 2017, to May 6, 2020. Results: Relative to unexposed children, the IRRs of injury hospitalizations for children with two SMI-affected parents, maternal SMI only, and paternal SMI only were 1.85 (95% CI, 1.38-2.48), 1.58 (95% CI, 1.48-1.68), and 1.34 (95% CI, 1.23-1.46), respectively. The IRRs of injury hospitalizations for maternal schizophrenia, bipolar disorder, and MDD were 2.09 (95% CI, 1.82-2.40), 1.77 (95% CI, 1.56-2.00), and 1.38 (95% CI, 1.26-1.50), respectively. The IRRs for paternal schizophrenia, bipolar disorder, and MDD were 1.39 (95% CI, 1.20-1.60), 1.61 (95% CI, 1.39-1.87), and 1.19 (95% CI, 1.05-1.36), respectively. The magnitude of excess risk was similar for children whose parent(s) experienced SMI diagnosed before and after childbirth. Conclusions: We found children with two SMI-affected parents or at least one parent with schizophrenia or bipolar disorder to be at greatest risk of severe injury requiring hospitalization. These parents may benefit from extra parenting support and injury prevention coaching.

Machine Learning Prediction of Suicide Risk Does Not Identify Patients Without Traditional Risk Factors

Objective: To determine whether predictions of suicide risk from machine learning models identify unexpected patients or patients without medical record documentation of traditional risk factors.

Methods: The study sample included 27,091,382 outpatient mental health (MH) specialty or general medical visits with a MH diagnosis for patients aged 11 years or older from January 1, 2009, to September 30, 2017. We used predicted risk scores of suicide attempt and suicide death, separately, within 90 days of visits to classify visits into risk score percentile strata. For each stratum, we calculated counts and percentages of visits with traditional risk factors, including prior self-harm diagnoses and emergency department visits or hospitalizations with MH diagnoses, in the last 3, 12, and 60 months.

Results: Risk-factor percentages increased with predicted risk scores. Among MH specialty visits, 66%, 88%, and 99% of visits with suicide attempt risk scores in the top 3 strata (respectively, 90th-95th, 95th-98th, and ≥ 98th percentiles) and 60%, 77%, and 93% of visits with suicide risk scores in the top 3 strata represented patients who had at least one traditional risk factor documented in the prior 12 months. Among general medical visits, 52%, 66%, and 90% of visits with suicide attempt risk scores in the top 3 strata and 45%, 66%, and 79% of visits with suicide risk scores in the top 3 strata represented patients who had a history of traditional risk factors in the last 12 months.

Conclusions: Suicide risk alerts based on these machine learning models coincide with patients traditionally thought of as high-risk at their high-risk visits.

Detecting and distinguishing indicators of risk for suicide using clinical records

Health systems are essential for suicide risk detection. Most efforts target people with mental health (MH) diagnoses, but this only represents half of the people who die by suicide. This study seeks to discover and validate health indicators of suicide death among those with, and without, MH diagnoses. This case-control study used statistical modeling with health record data on diagnoses, procedures, and encounters. The study included 3,195 individuals who died by suicide from 2000 to 2015 and 249,092 randomly selected matched controls, who were age 18+ and affiliated with nine Mental Health Research Network affiliated health systems. Of the 202 indicators studied, 170 (84%) were associated with suicide in the discovery cohort, with 148 (86%) of those in the validation cohort. Malignant cancer diagnoses were risk factors for suicide in those without MH diagnoses, and multiple individual psychiatric-related indicators were unique to the MH subgroup. Protective effects across MH-stratified models included diagnoses of benign neoplasms, respiratory infections, and utilization of reproductive services. MH-stratified latent class models validated five subgroups with distinct patterns of indicators in both those with and without MH. The highest risk groups were characterized via high utilization with multiple healthcare concerns in both groups. The lowest risk groups were characterized as predominantly young, female, and high utilizers of preventive services. Healthcare data include many indicators of suicide risk for those with and without MH diagnoses, which may be used to support the identification and understanding of risk as well as targeting of prevention in health systems.

Are People Who Die by Intentional Medication Poisoning Dispensed Those Medications in the Year Prior to Death?

OBJECTIVE

The rate of suicidal poisoning in the United States has increased substantially over the past 20 years. Understanding whether prescription medications used for self-poisoning were recently dispensed would help inform suicide prevention efforts. Alternatively, medications for self-poisoning could have been formerly dispensed or collected from friends, family, or illicit sources.

METHODS

Among those who died by intentional opioid and psychotropic poisonings, we conducted a descriptive study to determine what proportion had a recently filled prescription that could have been the means of suicide. Subjects were all people who died by intentional poisoning across nine health-care systems within the NIH-funded Mental Health Research Network.

RESULTS

Among the 3,300 people who died by suicide, 700 died by any poisoning and 194 died by intentional opioid or psychotropic/hypnotic medication poisoning. Among those who died by intentional opioid poisoning 73% were dispensed an opioid in the year prior. Among those who died by intentional psychotropic/hypnotic poisoning, 83% were dispensed any psychotropic and 61% were dispensed a hypnotic in prior year. Most people were continuously dispensed the same medications used in their intentional poisonings in the year prior to death.

CONCLUSIONS

Our results indicate that most medications used in suicidal overdose were likely recently dispensed. Therefore, future suicide prevention studies and prevention resources should focus on medication safety interventions such as lethal-means counseling for medication access, limiting quantities dispensed, opioid antagonists, and blister packs. HIGHLIGHTSUnderstanding whether medications used for self-poisoning were recently dispensed or formerly/never dispensed would help inform future studies and suicide prevention efforts.We found that most people who died by intentional poisoning with opioids or psychotropic/hypnotic medications received frequent dispensings of the medication used for self-poisoning in the year prior to death.Future suicide prevention studies and efforts should focus on medication safety interventions such as lethal-means counseling for medication access, limiting quantities dispensed, opioid antagonists, and blister packs.

Effect of Offering Care Management or Online Dialectical Behavior Therapy Skills Training vs Usual Care on Self-harm Among Adult Outpatients With Suicidal Ideation: A Randomized Clinical Trial

IMPORTANCE

People at risk of self-harm or suicidal behavior can be accurately identified, but effective prevention will require effective scalable interventions.

OBJECTIVE

To compare 2 low-intensity outreach programs with usual care for prevention of suicidal behavior among outpatients who report recent frequent suicidal thoughts.

DESIGN, SETTING, AND PARTICIPANTS

Pragmatic randomized clinical trial including outpatients reporting frequent suicidal thoughts identified using routine Patient Health Questionnaire depression screening at 4 US integrated health systems. A total of 18 882 patients were randomized between March 2015 and September 2018, and ascertainment of outcomes continued through March 2020.

INTERVENTIONS

Patients were randomized to a care management intervention (n = 6230) that included systematic outreach and care, a skills training intervention (n = 6227) that introduced 4 dialectical behavior therapy skills (mindfulness, mindfulness of current emotion, opposite action, and paced breathing), or usual care (n = 6187). Interventions, lasting up to 12 months, were delivered primarily through electronic health record online messaging and were intended to supplement ongoing mental health care.

MAIN OUTCOMES AND MEASURES

The primary outcome was time to first nonfatal or fatal self-harm. Nonfatal self-harm was ascertained from health system records, and fatal self-harm was ascertained from state mortality data. Secondary outcomes included more severe self-harm (leading to death or hospitalization) and a broader definition of self-harm (selected injuries and poisonings not originally coded as self-harm).

RESULTS

A total of 18 644 patients (9009 [48%] aged 45 years or older; 12 543 [67%] female; 9222 [50%] from mental health specialty clinics and the remainder from primary care) contributed at least 1 day of follow-up data and were included in analyses. Thirty-one percent of participants offered care management and 39% offered skills training actively engaged in intervention programs. A total of 540 participants had a self-harm event (including 45 deaths attributed to self-harm and 495 nonfatal self-harm events) over 18 months following randomization: 172 (3.27%) in care management, 206 (3.92%) in skills training, and 162 (3.27%) in usual care. Risk of fatal or nonfatal self-harm over 18 months did not differ significantly between the care management and usual care groups (hazard ratio [HR], 1.07; 97.5% CI, 0.84-1.37) but was significantly higher in the skills training group than in usual care (HR, 1.29; 97.5% CI, 1.02-1.64). For severe self-harm, care management vs usual care had an HR of 1.03 (97.5% CI, 0.71-1.51); skills training vs usual care had an HR of 1.34 (97.5% CI, 0.94-1.91). For the broader self-harm definition, care management vs usual care had an HR of 1.10 (97.5% CI, 0.92-1.33); skills training vs usual care had an HR of 1.17 (97.5% CI, 0.97-1.41).

CONCLUSIONS AND RELEVANCE

Among adult outpatients with frequent suicidal ideation, offering care management did not significantly reduce risk of self-harm, and offering brief dialectical behavior therapy skills training significantly increased risk of self-harm, compared with usual care. These findings do not support implementation of the programs tested in this study.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02326883.

Development of a Symptom-Focused Model to Guide the Prescribing of Antipsychotics in Children and Adolescents: Results of the First Phase of the Safer Use of Antipsychotics in Youth (SUAY) Clinical Trial

OBJECTIVE

To develop a new approach to prescribing guidelines as part of a pragmatic trial, Safer Use of Antipsychotics in Youth (SUAY; ClinicalTrials.gov Identifier: NCT03448575), which supports prescribers in delivering high-quality mental health care to youths.

METHOD

A nominal group technique was used to identify first- to nth-line treatments for target symptoms and potential diagnoses. The panel included US pediatricians, child and adolescent psychiatrists, and psychopharmacology experts. Meeting materials included information about Medicaid review programs, systematic reviews, prescribing guidelines, and a description of the pragmatic trial. Afterward, a series of 4 webinar discussions were held to achieve consensus on recommendations.

RESULTS

The panel unanimously agreed that the guideline should focus on target symptoms rather than diagnoses. Guidance included recommendations for first- to nth-line treatment of target mental health symptoms, environmental factors to be addressed, possible underlying diagnoses that should first be considered and ruled out, and general considerations for pharmacological and therapeutic treatments.

CONCLUSION

Prescribing guidelines are often ignored because they do not incorporate the real-world availability of first-line psychosocial treatments, comorbid conditions, and clinical complexity. Our approach addresses some of these concerns. If the approach proves successful in our ongoing pragmatic trial, Safer Use of Antipsychotics in Youth (SUAY), it may serve as a model to state Medicaid programs and health systems to support clinicians in delivering high-quality mental health care to youths.

CLINICAL TRIAL REGISTRATION INFORMATION

Safer Use of Antipsychotics in Youth; http://clinicaltrials.gov/; NCT03448575.

When Can We Rely on Real-World Evidence to Evaluate New Medical Treatments?

Concerns regarding both the limited generalizability and the slow pace of traditional randomized trials have led to calls for greater use of real-world evidence (RWE) in the evaluation of new treatments or products. The RWE label has been used to refer to a variety of departures from the methods of traditional randomized controlled trials. Recognizing this complexity and potential confusion, the National Academies of Science, Engineering, and Medicine convened a series of workshops to clarify and address questions regarding the use of RWE to evaluate new medical treatments. Those workshops identified three specific dimensions in which RWE studies might differ from traditional clinical trials: use of real-world data (data extracted from health system records or data captured by mobile devices), delivery of real-world treatment (open-label treatments delivered in community settings by community practitioners), and real-world treatment assignment (including nonrandomized comparisons and variations on random assignment such as before-after or stepped-wedge designs). For any RWE study, decisions regarding each of these dimensions depends on the specific research question, characteristics of the potential study settings, and characteristics of the settings where study results would be applied.

Prediction of suicidal behavior using self-reported suicidal ideation among patients with bipolar disorder

BACKGROUND

People with bipolar disorder have elevated suicide risk. We estimated the ability of the Patient Health Questionnaire (PHQ9) to predict suicide outcomes for outpatients with bipolar disorder.

METHODS

Visits by adults with bipolar disorder who completed a PHQ9 were identified using electronic health record (EHR) data. Bipolar diagnoses and suicide attempts were ascertained from EHR and claims data, and suicide deaths from state and federal records. Depression severity was assessed via the first eight items of the PHQ9, while suicidal ideation was assessed by the ninth item.

RESULTS

37,243 patients made 126,483 visits. Patients reported at least moderate symptoms of depression in 49% and suicidal ideation in 30% of visits. Risk of suicide attempt was 4.21% in the subsequent 90 days for those reporting nearly daily suicidal ideation compared to 0.74% in those reporting none. Patients with nearly daily suicidal ideation were 3.85 (95% CI 3.32-4.47) times more likely to attempt suicide and 13.78 (95% CI 6.56-28.94) times more likely to die by suicide in the subsequent 90 days than patients reporting none. Patients with self-harm in the last year were 8.86 (95% 7.84-10.02) times more likely to attempt suicide in the subsequent 90 days than those without.

LIMITATIONS

Our sample was limited to patients completing the PHQ9 and did not include data on some important social risk or protective factors.

CONCLUSIONS

The PHQ9 was a robust predictor of suicide. Suicidal ideation reported on the PHQ9 should be considered a strong indicator of suicide risk and prompt further evaluation.

Predicting suicide attempts and suicide deaths among adolescents following outpatient visits

BACKGROUND

Few studies report on machine learning models for suicide risk prediction in adolescents and their utility in identifying those in need of further evaluation. This study examined whether a model trained and validated using data from all age groups works as well for adolescents or whether it could be improved.

METHODS

We used healthcare data for 1.4 million specialty mental health and primary care outpatient visits among 256,823 adolescents across 7 health systems. The prediction target was 90-day risk of suicide attempt following a visit. We used logistic regression with least absolute shrinkage and selection operator (LASSO) and generalized estimating equations (GEE) to predict risk. We compared performance of three models: an existing model, a recalibrated version of that model, and a newly-learned model. Models were compared using area under the receiver operating curve (AUC), sensitivity, specificity, positive predictive value and negative predictive value.

RESULTS

The AUC produced by the existing model for specialty mental health visits estimated in adolescents alone (0.796; [0.789, 0.802]) was not significantly different than the AUC of the recalibrated existing model (0.794; [0.787, 0.80]) or the newly-learned model (0.795; [0.789, 0.801]). Predicted risk following primary care visits was also similar: existing (0.855; [0.844, 0.866]), recalibrated (0.85 [0.839, 0.862]), newly-learned (0.842, [0.829, 0.854]).

LIMITATIONS

The models did not incorporate non-healthcare risk factors. The models relied on ICD9-CM codes for diagnoses and outcome measurement.

CONCLUSIONS

Prediction models already in operational use by health systems can be reliably employed for identifying adolescents in need of further evaluation.

Utility of item 9 of the patient health questionnaire in the prospective identification of adolescents at risk of suicide attempt

INTRODUCTION

Previous studies report that item 9 of the Patient Health Questionnaire (PHQ9) is useful for stratifying risk of suicide attempt in adults. This study re-produced the utility of item 9 of PHQ9 in assessing risk of suicide attempt in adolescents.

MATERIALS AND METHODS

Individuals aged 13 to 17 years in 4 health systems with a diagnosis of depression and history of treatment were included. We estimated time to first observed fatal or non-fatal suicide attempt in the 2 years following completion of a PHQ9, stratified by response to item 9.

RESULTS

There were 51,807 PHQ9 questionnaires for 20,363 youth and 861 instances of suicide attempt. Cumulative probability of suicide attempt ranged from approximately 3.3% (95% CI, 3.0 to 3.5%) for those responding "not at all" on item 9 to 10.8% (95% CI, 9.2 to 12.4%) for those responding "nearly every day". These probabilities are more than 3 times higher than previously reported in adults.

CONCLUSION

PHQ item 9 is useful for stratifying risk of suicide attempt in the 2 years following completion of the questionnaire. Monitoring PHQ item 9 over time for patients in treatment for depression can be useful for population health management of adolescents with depression.

Predicting outcomes of psychotherapy for depression with electronic health record data

Predictive analytics with electronic health record (EHR) data holds promise for improving outcomes of psychiatric care. This study evaluated models for predicting outcomes of psychotherapy for depression in a clinical practice setting. EHR data from two large integrated health systems (Kaiser Permanente Colorado and Washington) included 5,554 new psychotherapy episodes with a baseline Patient Health Questionnaire (PHQ-9) score ≥ 10 and a follow-up PHQ-9 14–180 days after treatment initiation. Baseline predictors included demographics and diagnostic, medication, and encounter history. Prediction models for two outcomes—follow-up PHQ-9 score and treatment response (≥ 50% PHQ-9 reduction)—were trained in a random sample of 70% of episodes and validated in the remaining 30%. Two methods were used for modeling: generalized linear regression models with variable selection and random forests. Sensitivity analyses considered alternate predictor, outcome, and model specifications. Predictions of follow-up PHQ-9 scores poorly estimated observed outcomes (mean squared error = 31 for linear regression, 40 for random forest). Predictions of treatment response had low discrimination (AUC = 0.57 for logistic regression, 0.61 for random forest), low classification accuracy, and poor calibration. Sensitivity analyses showed similar results. We note that prediction model performance may vary for settings with different care or EHR documentation practices. In conclusion, prediction models did not accurately predict depression treatment outcomes despite using rich EHR data and advanced analytic techniques. Health systems should proceed cautiously when considering prediction models for psychiatric outcomes using baseline intake information. Transparent research should be conducted to evaluate performance of any model intended for clinical use.

Serious Suicide Attempts and Risk of Suicide Death

Background: In the US, more than one million people attempt suicide each year. History of suicide attempt is a significant risk factor for death by suicide; however, there is a paucity of data from the US general population on this relationship. Aim: The objective of this study was to examine suicide attempts needing medical attention as a risk for suicide death. Method: We conducted a case-control study involving eight US healthcare systems. A total of 2,674 individuals who died by suicide from 2000 to 2013 were matched to 267,400 individuals by year and location. Results: Prior suicide attempt associated with a medical visit increases risk for suicide death by 39.1 times, particularly for women (OR = 79.2). However, only 11.3% of suicide deaths were associated with an attempt that required medical attention. The association was the strongest for children 10-14 years old (OR = 98.0). Most suicide attempts were recorded during the 20-week period prior to death. Limitations: Our study is limited to suicide attempts for which individuals sought medical care. Conclusion: In the US, prior suicide attempt is associated with an increased risk of suicide death; the risk is high especially during the period immediately following a nonlethal attempt.

Zelen design clinical trials: why, when, and how

BACKGROUND

In 1979, Marvin Zelen proposed a new design for randomized clinical trials intended to facilitate clinicians' and patients' participation. The defining innovation of Zelen's proposal was random assignment of treatment prior to patient or participant consent. Following randomization, a participant would receive information and asked to consent to the assigned treatment.

METHODS

This narrative review examined recent examples of Zelen design trials evaluating clinical and public health interventions.

RESULTS

Zelen designs have often been applied to questions regarding real-world treatment or intervention effects under conditions of incomplete adherence. Examples include evaluating outreach or engagement interventions (especially for stigmatized conditions), evaluating treatments for which benefit may vary according to participant motivation, and situations when assignment to a control or usual care condition might prompt a disappointment effect. Specific practical considerations determine whether a Zelen design is scientifically appropriate or practicable. Zelen design trials usually depend on identifying participants automatically from existing records rather than by advertising, referral, or active recruitment. Assessments of baseline or prognostic characteristics usually depend on available records data rather than research-specific assessments. Because investigators must consider how exposure to treatments or interventions might bias ascertainment of outcomes, assessment of outcomes from routinely created records is often necessary. A Zelen design requires a waiver of the usual requirement for informed consent prior to random assignment of treatment. The Revised Common Rule includes specific criteria for such a waiver, and those criteria are most often met for evaluation of a low-risk and potentially beneficial intervention added to usual care. Investigators and Institutional Review Boards must also consider whether the scientific or public health benefit of a Zelen design trial outweighs the autonomy interests of potential participants. Analysis of Zelen trials compares outcomes according to original assignment, regardless of any refusal to accept or participate in the assigned treatment.

CONCLUSIONS

A Zelen design trial assesses the real-world consequences of a specific strategy to prompt or promote uptake of a specific treatment. While such trials are poorly suited to address explanatory or efficacy questions, they are often preferred for addressing pragmatic or policy questions.

Self-reported Access to Firearms Among Patients Receiving Care for Mental Health and Substance Use

Question

Did patients respond to a standard question about firearm access on a mental health questionnaire, and, if so, how did they respond?

Findings

In this cross-sectional study of 128 802 patients receiving care for mental health and substance use, 83% of primary care patients answered a standard question about firearm access and 21% reported access. In mental health clinics, 92% of patients answered the question and 15% reported access.

Meaning

In this study, most patients reported firearm access on standard questionnaires; this screening practice may improve efforts to identify and engage patients at risk of suicide in discussions about securing firearms.

Importance

Firearms are the most common method of suicide, one of the “diseases of despair” driving increased mortality in the US over the past decade. However, routine standardized questions about firearm access are uncommon, particularly among adult populations, who are more often asked at the discretion of health care clinicians. Because standard questions are rare, patterns of patient-reported access are unknown.

Objective

To evaluate whether and how patients self-report firearm access information on a routine mental health monitoring questionnaire and additionally to examine sociodemographic and clinical associations of reported access.

Design, Setting, and Participants

Cross-sectional study of patients receiving care for mental health and/or substance use in primary care or outpatient mental health specialty clinics of Kaiser Permanente Washington, an integrated health insurance provider and care delivery system.

Main Outcomes and Measures

Electronic health records were used to identify patients who completed a standardized self-reported mental health monitoring questionnaire after a single question about firearm access was added from January 1, 2016, through December 31, 2019. Primary analyses evaluated response (answered vs not answered) and reported access (yes vs no) among those who answered, separately for patients seen in primary care and mental health. These analyses also evaluated associations between patient characteristics and reported firearm access. Data analysis took place from February 2020 through May 2021.

Results

Among patients (n = 128 802) who completed a mental health monitoring questionnaire during the study period, 74.4% (n = 95 875) saw a primary care clinician and 39.3% (n = 50 631) saw a mental health specialty clinician. The primary care and mental health samples were predominantly female (63.1% and 64.9%, respectively) and White (75.7% and 77.0%), with a mean age of 42.8 and 51.1 years. In primary care, 83.4% of patients answered the question about firearm access, and 20.9% of patients who responded to the firearm question reported having access. In mental health, 91.8% of patients answered the question, and 15.3% reported having access.

Conclusions and Relevance

In this cross-sectional study of adult patients receiving care for mental health and substance use, most patients answered a question about firearm access on a standardized mental health questionnaire. These findings provide a critical foundation to help advance understanding of the utility of standardized firearm access assessment and to inform development of practice guidelines and recommendations. Responses to standard firearm access questions used in combination with dialogue and decision-making resources about firearm access and storage may improve suicide prevention practices and outcomes.

Evaluation of Electronic Health Record-Based Suicide Risk Prediction Models on Contemporary Data

BACKGROUND

Suicide risk prediction models have been developed by using information from patients' electronic health records (EHR), but the time elapsed between model development and health system implementation is often substantial. Temporal changes in health systems and EHR coding practices necessitate the evaluation of such models in more contemporary data.

OBJECTIVES

A set of published suicide risk prediction models developed by using EHR data from 2009 to 2015 across seven health systems reported c-statistics of 0.85 for suicide attempt and 0.83 to 0.86 for suicide death. Our objective was to evaluate these models' performance with contemporary data (2014-2017) from these systems.

METHODS

We evaluated performance using mental health visits (6,832,439 to mental health specialty providers and 3,987,078 to general medical providers) from 2014 to 2017 made by 1,799,765 patients aged 13+ across the health systems. No visits in our evaluation were used in the previous model development. Outcomes were suicide attempt (health system records) and suicide death (state death certificates) within 90 days following a visit. We assessed calibration and computed c-statistics with 95% confidence intervals (CI) and cut-point specific estimates of sensitivity, specificity, and positive/negative predictive value.

RESULTS

Models were well calibrated; 46% of suicide attempts and 35% of suicide deaths in the mental health specialty sample were preceded by a visit (within 90 days) with a risk score in the top 5%. In the general medical sample, 53% of attempts and 35% of deaths were preceded by such a visit. Among these two samples, respectively, c-statistics were 0.862 (95% CI: 0.860-0.864) and 0.864 (95% CI: 0.860-0.869) for suicide attempt, and 0.806 (95% CI: 0.790-0.822) and 0.804 (95% CI: 0.782-0.829) for suicide death.

CONCLUSION

Performance of the risk prediction models in this contemporary sample was similar to historical estimates for suicide attempt but modestly lower for suicide death. These published models can inform clinical practice and patient care today.

Weighing the Association Between BMI Change and Suicide Mortality

OBJECTIVE

Suicide rates continue to rise, necessitating the identification of risk factors. Obesity and suicide mortality rates have been examined, but associations among weight change, death by suicide, and depression among adults in the United States remain unclear.

METHODS

Data from 387 people who died by suicide in 2000-2015 with a recorded body mass index (BMI) in the first and second 6 months preceding their death ("index date") were extracted from the Mental Health Research Network. Each person was matched with five people in a control group (comprising individuals who did not die by suicide) by age, sex, index year, and health care site (N=1,935).

RESULTS

People who died by suicide were predominantly male (71%), White (69%), and middle aged (mean age=57 years) and had a depression diagnosis (55%) and chronic health issues (57%) (corresponding results for the control group: 71% male, 66% White, 14% with depression diagnosis, and 43% with chronic health issues; mean age=56 years). Change in BMI within the year before the index date statistically significantly differed between those who died by suicide (mean change=-0.72±2.42 kg/m²) and the control group (mean change=0.06±4.99 kg/m²) (p<0.001, Cohen's d=0.17). A one-unit BMI decrease was associated with increased risk for suicide after adjustment for demographic characteristics, mental disorders, and Charlson comorbidity score (adjusted odds ratio=1.11, 95% confidence interval=1.05-1.18, p<0.001). For those without depression, a BMI change was significantly associated with suicide (p<0.001).

CONCLUSIONS

An increased suicide mortality rate was associated with weight loss in the year before a suicide after analyses accounted for general and mental health indicators.

Racial/Ethnic Disparities in the Performance of Prediction Models for Death by Suicide After Mental Health Visits

IMPORTANCE

Clinical prediction models estimated with health records data may perpetuate inequities.

OBJECTIVE

To evaluate racial/ethnic differences in the performance of statistical models that predict suicide.

DESIGN, SETTING, AND PARTICIPANTS

In this diagnostic/prognostic study, performed from January 1, 2009, to September 30, 2017, with follow-up through December 31, 2017, all outpatient mental health visits to 7 large integrated health care systems by patients 13 years or older were evaluated. Prediction models were estimated using logistic regression with LASSO variable selection and random forest in a training set that contained all visits from a 50% random sample of patients (6 984 184 visits). Performance was evaluated in the remaining 6 996 386 visits, including visits from White (4 031 135 visits), Hispanic (1 664 166 visits), Black (578 508 visits), Asian (313 011 visits), and American Indian/Alaskan Native (48 025 visits) patients and patients without race/ethnicity recorded (274 702 visits). Data analysis was performed from January 1, 2019, to February 1, 2021.

EXPOSURES

Demographic, diagnosis, prescription, and utilization variables and Patient Health Questionnaire 9 responses.

MAIN OUTCOMES AND MEASURES

Suicide death in the 90 days after a visit.

RESULTS

This study included 13 980 570 visits by 1 433 543 patients (64% female; mean [SD] age, 42 [18] years. A total of 768 suicide deaths were observed within 90 days after 3143 visits. Suicide rates were highest for visits by patients with no race/ethnicity recorded (n = 313 visits followed by suicide within 90 days, rate = 5.71 per 10 000 visits), followed by visits by Asian (n = 187 visits followed by suicide within 90 days, rate = 2.99 per 10 000 visits), White (n = 2134 visits followed by suicide within 90 days, rate = 2.65 per 10 000 visits), American Indian/Alaskan Native (n = 21 visits followed by suicide within 90 days, rate = 2.18 per 10 000 visits), Hispanic (n = 392 visits followed by suicide within 90 days, rate = 1.18 per 10 000 visits), and Black (n = 65 visits followed by suicide within 90 days, rate = 0.56 per 10 000 visits) patients. The area under the curve (AUC) and sensitivity of both models were high for White, Hispanic, and Asian patients and poor for Black and American Indian/Alaskan Native patients and patients without race/ethnicity recorded. For example, the AUC for the logistic regression model was 0.828 (95% CI, 0.815-0.840) for White patients compared with 0.640 (95% CI, 0.598-0.681) for patients with unrecorded race/ethnicity and 0.599 (95% CI, 0.513-0.686) for American Indian/Alaskan Native patients. Sensitivity at the 90th percentile was 62.2% (95% CI, 59.2%-65.0%) for White patients compared with 27.5% (95% CI, 21.0%-34.7%) for patients with unrecorded race/ethnicity and 10.0% (95% CI, 0%-23.0%) for Black patients. Results were similar for random forest models, with an AUC of 0.812 (95% CI, 0.800-0.826) for White patients compared with 0.676 (95% CI, 0.638-0.714) for patients with unrecorded race/ethnicity and 0.642 (95% CI, 0.579-0.710) for American Indian/Alaskan Native patients and sensitivities at the 90th percentile of 52.8% (95% CI, 50.0%-55.8%) for White patients, 29.3% (95% CI, 22.8%-36.5%) for patients with unrecorded race/ethnicity, and 6.7% (95% CI, 0%-16.7%) for Black patients.

CONCLUSIONS AND RELEVANCE

These suicide prediction models may provide fewer benefits and more potential harms to American Indian/Alaskan Native or Black patients or those with undrecorded race/ethnicity compared with White, Hispanic, and Asian patients. Improving predictive performance in disadvantaged populations should be prioritized to improve, rather than exacerbate, health disparities.

Clinical risk prediction models and informative cluster size: Assessing the performance of a suicide risk prediction algorithm

Clinical visit data are clustered within people, which complicates prediction modeling. Cluster size is often informative because people receiving more care are less healthy and at higher risk of poor outcomes. We used data from seven health systems on 1,518,968 outpatient mental health visits from January 1, 2012 to June 30, 2015 to predict suicide attempt within 90 days. We evaluated true performance of prediction models using a prospective validation set of 4,286,495 visits from October 1, 2015 to September 30, 2017. We examined dividing clustered data on the person or visit level for model training and cross-validation and considered a within cluster resampling approach for model estimation. We evaluated optimism by comparing estimated performance from a left-out testing dataset to performance in the prospective dataset. We used two prediction methods, logistic regression with least absolute shrinkage and selection operator (LASSO) and random forest. The random forest model using a visit-level split for model training and testing was optimistic; it overestimated discrimination (area under the curve, AUC = 0.95 in testing versus 0.84 in prospective validation) and classification accuracy (sensitivity = 0.48 in testing versus 0.19 in prospective validation, 95th percentile cut-off). Logistic regression and random forest models using a person-level split performed well, accurately estimating prospective discrimination and classification: estimated AUCs ranged from 0.85 to 0.87 in testing versus 0.85 in prospective validation, and sensitivity ranged from 0.15 to 0.20 in testing versus 0.17 to 0.19 in prospective validation. Within cluster resampling did not improve performance. We recommend dividing clustered data on the person level, rather than visit level, to ensure strong performance in prospective use and accurate estimation of future performance at the time of model development.