Clinical risk prediction models and informative cluster size: Assessing the performance of a suicide risk prediction algorithm

Clinical visit data are clustered within people, which complicates prediction modeling. Cluster size is often informative because people receiving more care are less healthy and at higher risk of poor outcomes. We used data from seven health systems on 1,518,968 outpatient mental health visits from January 1, 2012 to June 30, 2015 to predict suicide attempt within 90 days. We evaluated true performance of prediction models using a prospective validation set of 4,286,495 visits from October 1, 2015 to September 30, 2017. We examined dividing clustered data on the person or visit level for model training and cross-validation and considered a within cluster resampling approach for model estimation. We evaluated optimism by comparing estimated performance from a left-out testing dataset to performance in the prospective dataset. We used two prediction methods, logistic regression with least absolute shrinkage and selection operator (LASSO) and random forest. The random forest model using a visit-level split for model training and testing was optimistic; it overestimated discrimination (area under the curve, AUC = 0.95 in testing versus 0.84 in prospective validation) and classification accuracy (sensitivity = 0.48 in testing versus 0.19 in prospective validation, 95th percentile cut-off). Logistic regression and random forest models using a person-level split performed well, accurately estimating prospective discrimination and classification: estimated AUCs ranged from 0.85 to 0.87 in testing versus 0.85 in prospective validation, and sensitivity ranged from 0.15 to 0.20 in testing versus 0.17 to 0.19 in prospective validation. Within cluster resampling did not improve performance. We recommend dividing clustered data on the person level, rather than visit level, to ensure strong performance in prospective use and accurate estimation of future performance at the time of model development.

When Can We Trust Real-World Data To Evaluate New Medical Treatments?

Concerns regarding both the limited generalizability and the slow pace of traditional randomized trials have led to calls for greater use of real‐world evidence (RWE) in the evaluation of new treatments or products. RWE studies often rely on real‐world data (RWD), including data extracted from healthcare records or data captured by mobile phones or other consumer devices. Global assessments of RWD sources are not helpful in assessing whether any specific RWD element is fit for any specific purpose. Instead, evidence generators and evidence consumers should clearly identify the specific health state or clinical phenomenon of interest and then consider each step between that clinical phenomenon and its representation in a research database. We propose specific questions regarding potential error or bias affecting each of those steps: Would a person experiencing this clinical phenomenon present for care in this setting or interact with this recording device? Would this clinical phenomenon be accurately recognized or assessed? How might the recording environment or tools affect accurate and consistent recording of this clinical phenomenon? Can data elements from different sources be harmonized, both technically (same format) and semantically (same meaning)? Can the original data elements be consistently reduced to a useful clinical phenotype? Addressing these questions requires a range of clinical, organizational, and technical expertise. Transparency regarding each step in the creation of RWD is essential if evidence consumers are to rely on RWE studies.

Reconciling Statistical and Clinicians' Predictions of Suicide Risk

Statistical models, including those based on electronic health records, can accurately identify patients at high risk for a suicide attempt or death, leading to implementation of risk prediction models for population-based suicide prevention in health systems. However, some have questioned whether statistical predictions can really inform clinical decisions. Appropriately reconciling statistical algorithms with traditional clinician assessment depends on whether predictions from these two methods are competing, complementary, or merely duplicative. In June 2019, the National Institute of Mental Health convened a meeting, "Identifying Research Priorities for Risk Algorithms Applications in Healthcare Settings to Improve Suicide Prevention." Here, participants of this meeting summarize key issues regarding the potential clinical application of suicide prediction models. The authors attempt to clarify the key conceptual and technical differences between traditional risk prediction by clinicians and predictions from statistical models, review the limited evidence regarding both the accuracy of and the concordance between these alternative methods of prediction, present a conceptual framework for understanding agreement and disagreement between statistical and clinician predictions, identify priorities for improving data regarding suicide risk, and propose priority questions for future research. Future suicide risk assessment will likely combine statistical prediction with traditional clinician assessment, but research is needed to determine the optimal combination of these two methods.

Brief Interventions via Electronic Health Record Messaging for Population-Based Suicide Prevention: Mixed Methods Pilot Study

Background

New opportunities to create and evaluate population-based selective prevention programs for suicidal behavior are emerging in health care settings. Standard depression severity measures recorded in electronic medical records (EMRs) can be used to identify patients at risk for suicide and suicide attempt, and promising interventions for reducing the risk of suicide attempt in at-risk populations can be adapted for web-based delivery in health care.

Objective

This study aims to evaluate a pilot of a psychoeducational program, focused on developing emotion regulation techniques via a web-based dialectical behavior therapy (DBT) skills site, including four DBT skills, and supported by secure message coaching, including elements of caring messages.

Methods

Patients were eligible based on the EMR-documented responses to the Patient Health Questionnaire indicating suicidal thoughts. We measured feasibility via the proportion of invitees who opened program invitations, visited the web-based consent form page, and consented; acceptability via qualitative feedback from participants about the DBT program; and engagement via the proportion of invitees who began DBT skills as well as the number of website visits for DBT skills and the degree of site engagement.

Results

A total of 60 patients were invited to participate. Overall, 93% (56/60) of the patients opened the invitation and 43% (26/60) consented to participate. DBT skills website users visited the home page on an average of 5.3 times (SD 6.0). Procedures resulted in no complaints and some participant feedback emphasizing the usefulness of DBT skills.

Conclusions

This study supports the potential of using responses to patient health questionnaires in EMRs to identify a high-risk population and offer key elements of caring messages and DBT adapted for a low-intensity intervention. A randomized trial evaluating the effectiveness of this program is now underway (ClinicalTrials.gov: NCT02326883).

When Are Treatment Blinding and Treatment Standardization Necessary in Real-World Clinical Trials?

Concerns regarding both the limited generalizability and the slow pace of traditional randomized trials have led to calls for greater use of real‐world evidence in the evaluation of new treatments or products. Real‐world clinical trials or pragmatic trials often differ from traditional clinical trials in the use of open‐label or nonblinded treatments delivered by real‐world clinicians in community practice settings. Blinding and standardization of treatment may sometimes be necessary for internal validity, but they may also obscure or distort meaningful differences between treatments. When investigators consider whether blinding of clinicians, patients, or assessors is necessary, we suggest they consider several specific questions: Will clinicians, patients, and assessors have expectations or preferences regarding benefits or adverse effects? How might those expectations affect treatment uptake, treatment adherence, or assessment of outcomes? Will expectations differ in the settings where trial results will be applied? How would blinding of treatment reduce biases? How would blinding obscure true differences between treatments? How would procedures necessary for blinding reduce acceptability or increase risk of trial participation? When investigators consider how strictly treatments should be standardized, we suggest they consider several specific questions: How would treatment effectiveness or safety vary according to clinician experience or expertise? What level of experience or expertise is available in potential trial settings and settings where trial results would be applied? Is some level of standardization necessary for valid inference? Considering any special vulnerabilities of the study population, is some level of standardization necessary to assure participant safety?

When Can Nonrandomized Studies Support Valid Inference Regarding Effectiveness or Safety of New Medical Treatments?

The randomized controlled trial (RCT) is the gold standard for evaluating the causal effects of medications. Limitations of RCTs have led to increasing interest in using real-world evidence (RWE) to augment RCT evidence and inform decision making on medications. Although RWE can be either randomized or nonrandomized, nonrandomized RWE can capitalize on the recent proliferation of large healthcare databases and can often answer questions that cannot be answered in randomized studies due to resource constraints. However, the results of nonrandomized studies are much more likely to be impacted by confounding bias, and the existence of unmeasured confounders can never be completely ruled out. Furthermore, nonrandomized studies require more complex design considerations which can sometimes result in design-related biases. We discuss questions that can help investigators or evidence consumers evaluate the potential impact of confounding or other biases on their findings: Does the design emulate a hypothetical randomized trial design? Is the comparator or control condition appropriate? Does the primary analysis adjust for measured confounders? Do sensitivity analyses quantify the potential impact of residual confounding? Are methods open to inspection and (if possible) replication? Designing a high-quality nonrandomized study of medications remains challenging and requires broad expertise across a range of disciplines, including relevant clinical areas, epidemiology, and biostatistics. The questions posed in this paper provide a guiding framework for assessing the credibility of nonrandomized RWE and could be applied across many clinical questions.

Evaluating the Cross-Cultural Measurement Invariance of the PHQ-9 between American Indian/Alaska Native Adults and Diverse Racial and Ethnic Groups

BACKGROUND

The Patient Health Questionnaire-9 (PHQ-9), a self-reported depression screening instrument for measurement-based care (MBC), may have cross-cultural measurement invariance (MI) with a regional group of American Indian/Alaska Native (AI/AN) and non-Hispanic White adults. However, to ensure health equity, research was needed on the cross-cultural MI of the PHQ-9 between other groups of AI/AN peoples and diverse populations.

METHODS

We assessed the MI of the one-factor PHQ-9 model and five previously identified two-factor models between non-Hispanic AI/AN adults (ages 18-64) from healthcare systems A (n=1,759) and B (n=2,701) using secondary data and robust maximum likelihood estimation. We then tested either fully or partially invariant models for MI between either combined or separate AI/AN groups, respectively, and Hispanic (n=7,974), White (n=7,974), Asian (n=6,988), Black (n=6,213), and Native Hawaiian/Pacific Islander (n=1,370) adults from healthcare system B. All had mental health or substance use disorder diagnoses and were seen in behavioral health or primary care from 1/1/2009-9/30/2017.

RESULTS

The one-factor PHQ-9 model was partially invariant, with two-factor models partially, or in one case fully, invariant between AI/AN groups. The one-factor model and three two-factor models were partially invariant between all seven groups, while a two-factor model was fully invariant and another partially invariant between a combined AI/AN group and other racial and ethnic groups.

CONCLUSIONS

Achieving health equity in MBC requires ensuring the cross-cultural validity of measurement tools. Before comparing mean scores, PHQ-9 models should be assessed for individual racial and ethnic group fit for adults with mental health or substance use disorders.

Psychiatric Symptoms in Multiple Chemical Sensitivity

Neuropsychiatric symptoms are among the most prominent manifestations of generalized chemical sensitivity. Patients, clinicians, and researchers are in agreement that symptoms such as depression, irritability, and mood instability are prominent among the distressing and disabling symptoms occurring in response to low-level chemical exposure. Beyond that point, however, agreement is difficult. The pathophysiology and clinical management of these symptoms remain quite controversial.

This paper will review available data on the prevalence and form of psychiatric symptoms among those suffering from multiple chemical sensitivity. Various models explaining the relationship of psychiatric symptoms to chemical sensitivity will be discussed. Finally, the implications of these models for clinical management and future research will be reviewed.

What health records data are required for accurate prediction of suicidal behavior?

OBJECTIVE

The study sought to evaluate how availability of different types of health records data affect the accuracy of machine learning models predicting suicidal behavior.

MATERIALS AND METHODS

Records from 7 large health systems identified 19 061 056 outpatient visits to mental health specialty or general medical providers between 2009 and 2015. Machine learning models (logistic regression with penalized LASSO [least absolute shrinkage and selection operator] variable selection) were developed to predict suicide death (n = 1240) or probable suicide attempt (n = 24 133) in the following 90 days. Base models were used only historical insurance claims data and were then augmented with data regarding sociodemographic characteristics (race, ethnicity, and neighborhood characteristics), past patient-reported outcome questionnaires from electronic health records, and data (diagnoses and questionnaires) recorded during the visit.

RESULTS

For prediction of any attempt following mental health specialty visits, a model limited to historical insurance claims data performed approximately as well (C-statistic 0.843) as a model using all available data (C-statistic 0.850). For prediction of suicide attempt following a general medical visit, addition of data recorded during the visit yielded a meaningful improvement over a model using all data up to the prior day (C-statistic 0.853 vs 0.838).

DISCUSSION

Results may not generalize to setting with less comprehensive data or different patterns of care. Even the poorest-performing models were superior to brief self-report questionnaires or traditional clinical assessment.

CONCLUSIONS

Implementation of suicide risk prediction models in mental health specialty settings may be less technically demanding than expected. In general medical settings, however, delivery of optimal risk predictions at the point of care may require more sophisticated informatics capability.

Suicide Behavior Following PHQ-9 Screening Among Individuals With Substance Use Disorders

OBJECTIVES

Individuals with substance use disorders (SUD) are at risk for suicide, but no studies have assessed whether routinely administered screeners for suicidal ideation accurately identify outpatients with SUD who are at risk for suicide attempt or death.

METHODS

Data from more than 186,000 visits by over 55,000 patients with mental health and SUD diagnoses receiving care in 7 health systems were analyzed to determine whether responses to item 9 of the 9-item Patient Health Questionnaire, which assesses frequency of thoughts of death and self-harm, are associated with suicide outcomes after an outpatient visit. Odds of suicide attempt or death were computed using generalized estimating equations.

RESULTS

In bivariate analyses, a nearly 5-fold risk was observed for patients answering "nearly every day" relative to "not at all" among individuals who made a suicide attempt within 90 days (4.9% vs 1.1%; χ2 = 1151, P < 0.0001). At nearly half of visits (46%) followed by a suicide attempt within 90 days, patients responded "not at all." In logistic models, compared to "not at all," all other responses were associated with higher odds of suicide attempt or death within 90 days. Fully adjusted models attenuated results but odds of suicide attempt (AOR = 3.24, CI: 2.69-3.91) and suicide death (AOR = 5.67, CI: 2.0-16.1) remained high for those reporting "nearly every day."

CONCLUSIONS

In people with SUD, increasing Patient Health Questionnaire item 9 response predicts increased risk of subsequent suicidal behavior and should prompt intervention. However, clinicians should realize that those reporting "not at all" are not immune from subsequent suicide risk.

An implementation evaluation of "Zero Suicide" using normalization process theory to support high-quality care for patients at risk of suicide

Background

Suicide rates continue to rise across the United States, galvanizing the need for increased suicide prevention and intervention efforts. The Zero Suicide (ZS) model was developed in response to this need and highlights four key clinical functions of high-quality health care for patients at risk of suicide. The goal of this quality improvement study was to understand how six large health care systems operationalized practices to support these functions-identification, engagement, treatment and care transitions.

Methods

Using a key informant interview guide and data collection template, researchers who were embedded in each health care system cataloged and summarized current and future practices supporting ZS, including, (1) the function addressed; (2) a description of practice intent and mechanism of intervention; (3) the target patient population and service setting; (4) when/how the practice was (or will be) implemented; and (5) whether/how the practice was documented and/or measured. Normalization process theory (NPT), an implementation evaluation framework, was applied to help understand how ZS had been operationalized in routine clinical practices and, specifically, what ZS practices were described by key informants (coherence), the current state of norms/conventions supporting these practices (cognitive participation), how health care teams performed these practices (collective action), and whether/how practices were measured when they occurred (reflexive monitoring).

Results

The most well-defined and consistently measured ZS practices (current and future) focused on the identification of patients at high risk of suicide. Stakeholders also described numerous engagement and treatment practices, and some practices intended to support care transitions. However, few engagement and transition practices were systematically measured, and few treatment practices were designed specifically for patients at risk of suicide.

Conclusions

The findings from this study will support large-scale evaluation of the effectiveness of ZS implementation and inform recommendations for implementation of high-quality suicide-related care in health care systems nationwide.

Safer use of antipsychotics in youth (SUAY) pragmatic trial protocol

BACKGROUND

Programs such as the Pediatric Access Line in Washington state have shown decreases in antipsychotic medication use by youth with non-psychotic disorders. Program outcomes have been studied with observational designs. This manuscript describes the protocol for Targeted and Safer Use of Antipsychotics in Youth (SUAY), a randomized controlled trial of psychiatrist review of prescriptions and facilitated access to psychosocial care. The aim of the intervention is to reduce the number of person-days of antipsychotic use among participants.

METHODS

Recruitment occurs at 4 health systems. Targeted enrollment is 800 youth aged 3-17 years. Clinicians are block randomized to intervention versus usual care prior to the study. Youth are nested within the arm of the prescribing clinician. Clinicians in the intervention group receive an EHR-based best practice alert with options to expedite access to psychosocial care and all medication orders are reviewed by a child and adolescent psychiatrist with feedback provided to the prescriber. The primary outcome is person-days of antipsychotic medication use in the 6 months following the initial order. All randomized individuals contribute data regardless of their level of participation (including declining all services).

DISCUSSION

The trial has been approved by the institutional review boards at each of the 4 sites. The intervention has 4 novel design features including automated recruitment using a best practice alert, psychiatrist medication order review and consultation, telephone navigation to psychosocial care, and telemental health visits. Recruitment began in March of 2018 and will be completed in June 2020. Follow-up will be completed December 31, 2020.

TRIAL REGISTRATION

Clinicaltrials.gov, NCT03448575.

Association of Parental Mental Illness With Child Injury Occurrence, Hospitalization, and Death During Early Childhood

IMPORTANCE

Injury is a leading cause of childhood morbidity and mortality worldwide. Serious mental illness (SMI) is a major contributor to the global burden of disease.

OBJECTIVE

To compare injury event rates in children from birth to 5 years of age among Taiwanese children with and without parents with SMI, including schizophrenia, bipolar disorder, and major depressive disorder.

DESIGN, SETTING, AND PARTICIPANTS

This population-based, retrospective cohort study of an 11-year Taiwanese birth cohort used data from the Taiwan National Health Insurance Research Database (covering 99% of Taiwanese citizens), the Maternal and Child Health Database, and birth and death certificate databases. The study included 1 999 322 singletons with Taiwanese citizenship born from January 1, 2004, to December 31, 2014, and followed up from birth to their fifth birthday, December 31, 2014, or the date of death, yielding a total of 7 741 026 person-years. Data analysis was performed from April 20, 2017, to September 24, 2019.

EXPOSURES

Physician-diagnosed parental SMI defined using outpatient and inpatient records from 6 years before the child's birth to 5 years after delivery.

MAIN OUTCOME AND MEASURES

Rates of medically attended injury events, injury hospitalization, and injury death retrieved from outpatient records, inpatient records, and death certificates. Generalized estimating equation for log-linear models estimated injury incidence rate ratios (IRRs) comparing parental SMI-exposed children and unexposed children.

RESULTS

The study cohort included 1 999 322 singletons (52.1% males without parental SMI and 52.2% males with parental SMI). Incidence rates of child injury-related outcomes were higher among children exposed to parental SMI (294.8 injury events per 1000 person-years) compared with children who were unexposed (256.1 injury events per 1000 person-years). After adjustment for sociodemographic factors, children with parental SMI had higher rates of injury events (IRR, 1.14; 95% CI, 1.13-1.15), injury hospitalization (IRR, 1.49; 95% CI, 1.42-1.57), and injury death (IRR, 1.82; 95% CI, 1.38-2.39) compared with unexposed children. The results were confirmed in sensitivity analyses. Appendicitis, a negative control outcome, was not associated with parental SMI (IRR, 1.10; 95% CI, 0.94-1.28). In addition, children with and without parental SMI had similar patterns of preventive health care. The mean (SD) number of prenatal visits was 8.09 (2.50) for children with parental SMI and 8.17 (2.47) among unaffected children. The mean (SD) number of well-child visits was 5.70 (2.24) for children with parental SMI and 5.80 (2.21) among unaffected children.

CONCLUSIONS AND RELEVANCE

In this study, children with parental SMI had increased risk of injury, particularly serious injury. Excess risk may be reduced by providing effective mental health treatment, parenting support, and home safety education to parents with SMI who are raising young children.

Harmonized Outcome Measures for Use in Depression Patient Registries and Clinical Practice

Major depressive disorder is a common mental health condition that affects an estimated 16.2 million adults and 3.1 million adolescents in the United States. Yet, a lack of uniformity remains in measurements and monitoring for depression both in clinical practice and in research settings. This project aimed to develop a minimum set of standardized outcome measures relevant to both patients and clinicians that can be collected in depression registries and clinical practice. Twenty-nine depression registries and related data collection efforts were identified and invited to submit outcome measures. Additional measures were identified through literature searches and reviews of quality measures. A multistakeholder panel representing clinicians; payers; government agencies; industry; and medical specialty, health care quality, and patient advocacy organizations categorized the 27 identified measures using the Agency for Healthcare Research and Quality's supported Outcome Measures Framework. The panel identified 10 broadly relevant measures and harmonized definitions for these measures through in-person and virtual meetings. The harmonized measures represent a minimum set of outcomes that are relevant to clinicians and patients and appropriate for use in depression research and clinical practice. Routine and consistent collection of these measures in registries and other systems would support creation of a national research infrastructure to efficiently address new questions, improve patient management and outcomes, and facilitate care coordination.

Defining Success in Measurement-Based Care for Depression: A Comparison of Common Metrics

OBJECTIVE

The National Committee for Quality Assurance recommends response and remission as indicators of successful depression treatment for the Healthcare Effectiveness and Data Information Set. Effect size and severity-adjusted effect size (SAES) offer alternative metrics. This study compared measures and examined the relationship between baseline symptom severity and treatment success.

METHODS

Electronic records from two large integrated health systems (Kaiser Permanente Colorado and Washington) were used to identify 5,554 new psychotherapy episodes with a baseline Patient Health Questionnaire (PHQ-9) score of ≥10 and a PHQ-9 follow-up score from 14-180 days after treatment initiation. Treatment success was defined for four measures: response (≥50% reduction in PHQ-9 score), remission (PHQ-9 score <5), effect size ≥0.8, and SAES ≥0.8. Descriptive analyses examined agreement of measures. Logistic regression estimated the association between baseline severity and success on each measure. Sensitivity analyses evaluated the impact of various outcome definitions and loss to follow-up.

RESULTS

Effect size ≥0.8 was most frequently attained (72% across sites), followed by SAES ≥0.8 (66%), response (46%), and remission (22%). Response was the only measure not associated with baseline PHQ-9 score. Effect size ≥0.8 favored episodes with a higher baseline PHQ-9 score (odds ratio [OR]=2.3, p<0.001, for 10-point difference in baseline PHQ-9 score), whereas SAES ≥0.8 (OR=0.61, p<0.001) and remission (OR=0.43, p<0.001) favored episodes with lower baseline scores.

CONCLUSIONS

Response is preferable for comparing treatment outcomes, because it does not favor more or less baseline symptom severity, indicates clinically meaningful improvement, and is transparent and easy to calculate.

Using predictive analytics to improve pragmatic trial design

Clinical trials embedded in health systems can randomize large populations using automated data sources to determine trial eligibility and assess outcomes. The suicide prevention outreach trial used real-world data for trial design and randomized 18,868 individuals in four health systems using patient-reported thoughts of death or self-harm (Patient Health Questionnaire item 9). This took 3.5 years. We consider if using predictive analytics, that is, suicide risk estimates based on prediction models, could improve trial "efficiency." We used data on mental health outpatient visits between 1 January 2009 and 30 September 2017 in seven health systems (HealthPartners; Henry Ford Health System; and Colorado, Hawaii, Northwest, Southern California, and Washington Kaiser Permanente regions). We used a suicide risk prediction model developed in these same systems. We compared five trial designs with different eligibility criteria: a response of a 2 or 3 on Patient Health Questionnaire item 9, a response of a 3, suicide risk score above 90th, 95th, or 99th percentile. We compared the sample that met each criterion, 90-day suicide attempt rate following first eligible visit, and necessary sample sizes to detect a 15%, 25%, and 35% relative reduction in the suicide attempt rate, assuming 90% power, for each eligibility criterion. Our sample included 24,355,599 outpatient visits. Despite wide-spread use of Patient Health Questionnaire, 21,026,985 (86.3%) visits did not have a recorded Patient Health Questionnaire. Of the 2,928,927 individuals in our sample, 109,861 had a recorded Patient Health Questionnaire item 9 response of a 2 or 3 over the study years with a 1.40% 90-day suicide attempt rate and 50,047 had a response of a 3 (suicide attempt rate 1.98%). More patients met criteria requiring a certain risk score or higher: 331,273 had a 90th percentile risk score or higher (suicide attempt rate: 1.36%); 182,316 a 95th percentile or higher (suicide attempt rate 2.16%), and 78,655 a 99th percentile or higher (suicide attempt rate: 3.95%). Eligibility criterion of a Patient Health Questionnaire item 9 response of a 2 or 3 would require randomizing 44,081 individuals (40.2% of eligible population in our sample); eligibility criterion of a 3 would require 31,024 individuals (62.0% of eligible population). Eligibility criterion of a suicide risk score of 90th percentile or higher would require 45,675 individuals (13.8% of eligible population), 95th percentile 28,699 individuals (15.7% of eligible population), and 99th percentile 15,509 (19.7% of eligible population). A suicide risk prediction calculator could improve trial "efficiency"; identifying more individuals at increased suicide risk than relying on patient-report. It is an open scientific question if individuals identified using predictive analytics would respond differently to interventions than those identified by more traditional means.

Association between suicide death and concordance with benzodiazepine treatment guidelines for anxiety and sleep disorders

OBJECTIVE

Guidelines for management of anxiety and sleep disorders emphasize antidepressant medications and/or psychotherapy as first/second-line and benzodiazepines as third-line treatments. We evaluated the association between suicide death and concordance with benzodiazepine guidelines.

METHODS

Retrospective case-control study of patients with anxiety and/or sleep disorders from health systems across 8 U.S. states within the Mental Health Research Network. Suicide death cases were matched to controls on year and health system. Appropriate benzodiazepine prescribing defined as: no monotherapy, no long duration, and/or age < 65 years. The association between guideline concordance and suicide death was evaluated, adjusting for diagnostic and treatment covariates.

RESULTS

Sample included 6960 patients with anxiety disorders (2363 filled benzodiazepine) and 6215 with sleep disorders (1237 filled benzodiazepine). Benzodiazepine guideline concordance was associated with reduced odds for suicide in patients with anxiety disorders (OR = 0.611, 95% CI = 0.392-0.953, p = 0.03) and was driven by shorter duration of benzodiazepine use with concomitant psychotherapy or antidepressant medication. The association of benzodiazepine guideline concordance with suicide death did not meet statistical significance in the sleep disorder group (OR = 0.413, 95% CI = 0.154-1.11, p = 0.08).

CONCLUSIONS

We found reduced odds for suicide in those with anxiety disorders who filled benzodiazepines in short-moderate duration with concomitant psychotherapy or antidepressant treatment.

Healthcare utilization among patients with psychiatric hospitalization admitted through the emergency department (ED): A claims-based study

OBJECTIVE

To characterize the US national level healthcare utilization patterns of patients with commercial insurance plans before and after a psychiatric hospitalization admitted through the emergency department (ED) using insurance claims data.

METHOD

We identified 34,250 patients from multiple commercial health insurance providers across the US who meet our eligibility criteria. We summarized their healthcare encounters and used logistic regression models to study the patterns of healthcare utilization including prior visits, outpatient follow-ups, and hospital- or ED-readmissions.

RESULTS

Suicidal ideation was highly prevalent at the time of the index event (29.88%). Almost half of the patients (48.28%) had healthcare encounters with the same primary diagnosis one year before admission, about 5% had outpatient follow-ups or were readmitted to the hospital or ED 7 days post discharge. The post 30-day follow-ups and readmission rates were slightly higher. In general, older patients were less likely to have prior visits, follow-ups, or readmissions, and patients with SUDs, specifically alcohol dependence, opioid dependence/abuse, and stimulant dependence, were more likely to have outpatient follow-ups.

CONCLUSION

Patterns of patients' prior visits, follow-ups, and readmissions varied by demographics and psychiatric comorbidity. Additional studies are needed to further explain the spatial variations of utilization patterns.

Principles and procedures for data and safety monitoring in pragmatic clinical trials

Background

All clinical trial investigators have ethical and regulatory obligations to monitor participant safety and trial integrity. Specific procedures for meeting these obligations, however, may differ substantially between pragmatic trials and traditional explanatory clinical trials.

Methods/Results

Appropriate monitoring of clinical trials typically includes assessing rate of recruitment or enrollment; monitoring safe and effective delivery of study treatments; assuring that study staff act to minimize risks; monitoring quality and timeliness of study data; and considering interim analyses for early detection of benefit, harm, or futility. Each of these responsibilities applies to pragmatic clinical trials. Just as design of pragmatic trials typically involves specific and necessary departures from methods of explanatory clinical trials, appropriate monitoring of pragmatic trials typically requires specific departures from monitoring procedures used in explanatory clinical trials. We discuss how specific aspects of pragmatic trial design and operations influence selection of monitoring procedures and illustrate those choices using examples from three ongoing pragmatic trials conducted by the Mental Health Research Network.

Conclusions

Pragmatic trial investigators should not routinely adopt monitoring procedures used in explanatory clinical trials. Instead, investigators should consider core principles of trial monitoring and design monitoring procedures appropriate for each pragmatic trial.

Patterns of Health Care Utilization Before First Episode Psychosis in Racial and Ethnic Groups

Objective

To compare patterns of health care utilization associated with first presentation of psychosis among different racial and ethnic groups of patients.

Design

The study was a retrospective observational design.

Setting

The study was conducted in five health care systems in the western United States. All sites were also part of the National Institute of Mental Health-funded Mental Health Research Network (MHRN).

Participants

Patients (n = 852) were aged 15 - 59 years (average 26.9 ± 12.2 years), 45% women, and primarily non-Hispanic White (53%), with 16% Hispanic, 10% non-Hispanic Black, 6% Asian, 1% Native Hawaiian/Pacific Islander, 1% Native American/ Alaskan Native, and 12% unknown race/ethnicity.

Main Outcome Measures

Variables examined were patterns of health care utilization, type of comorbid mental health condition, and type of treatment received in the three years before first presentation of psychosis.

Methods

Data abstracted from electronic medical records and insurance claims data were organized into a research virtual data warehouse (VDW) and used for analysis.

Results

Compared with non-Hispanic Whites, Asian patients (16% vs 34%; P=.007) and non-Hispanic Black patients (20% vs 34%; P=.009) were less likely to have a visit with specialty mental health care before their first presentation of psychosis.

Conclusions

Early detection of first episode psychosis should start with wider screening for symptoms outside of any indicators for mental health conditions for non-Hispanic Black and Asian patients.