A Review of the Impact of Exclusion Criteria on the Generalizability of Schizophrenia Treatment Research

A literature review of the impact of exclusion criteria on generalizability of clinical trial findings to patients with chronic pain

The ability of clinical trials to inform the care of chronic pain may be limited if only an unrepresentative subset of patients are allowed to enroll. We summarize and report new insights on published studies that report on how trial exclusions affect the generalizability of their results. We conducted a PubMed search on the following terms: (("eligibility criteria" AND generalizability) OR ("exclusion criteria" AND generalizability) OR "exclusion criteria"[ti] OR "eligibility criteria"[ti]) AND pain. We only considered studies relevant if they analyzed data on (1) the prevalence and nature of exclusion criteria or (2) the impact of exclusion criteria on sample representativeness or study results. The 4 articles that were identified reported differences in patients who were included and excluded in different clinical trials: excluded patients were older, less likely to have a paid job, had more functional limitations at baseline, and used strong opioids more often. The clinical significance of these differences remains unclear. The pain medicine literature has very few published studies on the prevalence and impact of exclusion criteria, and the outcomes of excluded patients are rarely tracked. The frequent use of psychosocial exclusions is especially compromising to generalizability because chronic pain commonly co-occurs with psychiatric comorbidities. Inclusion of more representative patients in research samples can reduce recruitment barriers and broaden the generalizability of findings in patients with chronic pain. We also call for more studies that examine the use of exclusion criteria in chronic pain trials to better understand their implications.

Symptom monitoring based on digital data collection during inpatient treatment of schizophrenia spectrum disorders - A feasibility study

Digital acquisition of patients' self-reports on individual risk factors and symptom severity represents a promising, cost-efficient, and increasingly prevalent approach for standardized data collection in psychiatric clinical routine. Yet, studies investigating digital data collection in patients with a schizophrenia spectrum disorder (PSSDs) are scarce. The objective of this study was to explore the feasibility of digitally acquired self-report assessments of risk and symptom profiles at the time of admission into inpatient treatment in an age-representative sample of hospitalized PSSDs. We investigated the required support, the data entry pace, and the subjective user experience. Findings were compared with those of patients with an affective disorder (PADs). Of 82 PSSDs who were eligible for inclusion, 59.8% (n=49) agreed to participate in the study, of whom 54.2% (n=26) could enter data without any assistance. Inclusion rates, drop-out rates, and subjective experience ratings did not differ between PSSDs and PADs. Patients reported high satisfaction with the assessment. PSSDs required more support and time for the data entry than PADs. Our results indicate that digital data collection is a feasible and well-received method in PSSDs. Future clinical and research efforts on digitized assessments in psychiatry should include PSSDs and offer support to reduce digital exclusion.

Acknowledging Patient Heterogeneity in Economic Evaluations in Schizophrenia: A Systematic Review

OBJECTIVES

Schizophrenia is a severe mental illness with heterogeneous etiology, range of symptoms, and course of illness. Cost-effectiveness analysis often applies averages from populations, which disregards patient heterogeneity even though there are a range of methods available to acknowledge patient heterogeneity. This review evaluates existing economic evaluations of interventions in schizophrenia to understand how patient heterogeneity is currently reflected in economic evaluation.

METHODS

Electronic searches of MEDLINE, Embase, and PsycINFO via Ovid and the Health Technology Assessment database were run to identify full economic evaluations of interventions aiming to reduce the symptoms associated with schizophrenia. Two levels of screening were used, and explicit inclusion criteria were applied. Prespecified data extraction and critical appraisal were performed.

RESULTS

Seventy-six relevant studies were identified. More than half (41 of 76) of the articles acknowledged patient heterogeneity in some way through discussion or methods. There was a range of patient characteristics considered, including demographics and socioeconomic factors (eg, age, educational level, ethnicity), clinical characteristics (eg, symptom severity, comorbidities), and preferences (eg, preferences related to outcomes or symptoms). Subgroup analyses were rarely reported (8 of 76).

CONCLUSIONS

Patient heterogeneity was frequently mentioned in studies but was rarely thoroughly investigated in the identified economic evaluations. When investigated, included patient characteristics and methods were found to be heterogeneous. Understanding and acknowledging patient heterogeneity may alter the conclusions of cost-effectiveness evaluations; subsequently, we would encourage further research in this area.

The Danish OPUS Early Intervention Services for First-Episode Psychosis: A Phase 4 Prospective Cohort Study With Comparison of Randomized Trial and Real-World Data

OBJECTIVE

The Danish OPUS trial showed significant efficacy of early intervention services for first-episode schizophrenia spectrum disorders compared with standard treatment, leading to implementation of the OPUS intervention in clinical practice. The authors sought to determine whether the effectiveness of OPUS treatment in real-world clinical practice is comparable to the efficacy seen in the trial.

METHODS

The study compared patients who received OPUS treatment as part of the original randomized trial to those who received standard treatment in the trial (the control group) and those who received OPUS treatment after it was implemented in Denmark. The authors investigated whether the three groups differed on register-based outcomes, such as use of secondary health care, functional outcomes, and death. Analyses were adjusted for relevant confounders.

RESULTS

Compared with trial study participants, patients who received OPUS treatment after implementation (N=3,328) had a tendency toward lower mortality (hazard ratio=0.60, 95% CI=0.33, 1.09), fewer and shorter psychiatric admissions, and possibly fewer filled prescriptions of antipsychotics and other psycholeptics after 4 or 5 years. While at first less likely to be working or studying, patients who received postimplementation OPUS treatment eventually had higher odds of working than did those in the OPUS trial (after 5 years, odds ratio=1.49, 95% CI=1.07, 2.09). The odds of being in a couple relationship were also higher among patients in the postimplementation group than those in the trial. Other outcomes showed less clear associations with treatment group. Generally, the control group in the trial fared worse than both of the OPUS treatment groups.

CONCLUSIONS

Not only did OPUS treatment maintain its efficacy after it was implemented as a standard treatment, it paralleled or surpassed many of the effects observed when the OPUS intervention was delivered in a randomized trial. The study results provide further evidence in support of implementation and funding of early intervention services worldwide.

Inclusion of Cannabis Users in Alcohol Research Samples: Screening In, Screening Out, and Implications

BACKGROUND

Alcohol and cannabis are frequently co-used, as 20-50% of those who drink alcohol report co-using cannabis. This study is based on the argument that alcohol researchers should enroll cannabis users in human laboratory studies of alcohol use disorder (AUD) to strengthen generalizability. This study examines how heavy drinking cannabis users differ from non-cannabis using heavy drinkers.

METHODS

In a community sample of non-treatment-seeking heavy drinkers (n = 551, 35% female), cannabis users were identified through: (a) self-reported cannabis use in the past 6 months and (b) positive urine toxicology test for tetrahydrocannabinol (THC). Cannabis users, identified as described previously, were compared with non-cannabis users on demographic and clinical characteristics.

RESULTS

Those who endorsed cannabis use in the past 6 months reported more binge drinking days. Participants who tested positive for THC had higher Alcohol Use Disorder Identification Test scores and more binge drinking days. Younger age and being a tobacco smoker were associated with an increased likelihood of cannabis use in the past 6 months, whereas male gender and being a tobacco use were associated with a greater likelihood of testing positive for THC. Individuals with cannabis use disorder (CUD) endorsed more depression and anxiety and had higher AUD symptom counts than cannabis users without CUD.

CONCLUSIONS

The inclusion of cannabis users in AUD samples allows for increased clinical severity. Excluding cannabis users from AUD studies may limit representativeness and expend unnecessary study resources. Lastly, tobacco use may explain a large portion of the effects of cannabis use on sample characteristics.

SHORT SUMMARY

Alcohol and cannabis are frequently co-used substances. In a sample of non-treatment-seeking heavy drinkers (n = 551, 35% female), cannabis users reported higher alcohol use and higher likelihood of tobacco use than non-cannabis users. Including cannabis users in alcohol research studies will improve representativeness and likely increase clinical severity.

Applying a Transdiagnostic Cognitive-Behavioral Treatment to Adolescents at High Risk for Serious Mental Illness: Rationale and Preliminary Findings

Given the chronic and deleterious course of serious mental illness (SMI; schizophrenia and bipolar disorder), significant efforts have been undertaken to improve prediction of SMI and provide treatment for adolescents in the early, putatively prodromal stage of these illnesses. While risk assessments and disorder-specific treatments for adolescents at risk for SMI have shown some efficacy, significant issues remain around disorder-specific treatments for these youth. There is substantial heterogeneity of psychopathology within adolescents at high risk for SMI that leads to many false-positives and varying diagnostic outcomes. As a result, initial treatment focusing on broad symptoms and skills has been proposed in place of disorder-specific treatments. We discuss the rationale for providing an already-developed and empirically supported transdiagnostic treatment for emotional disorders (termed the Unified Protocol) as a first-line staging of treatment for adolescents experiencing early SMI symptoms. Additionally, we outline the open trial we are piloting using this transdiagnostic treatment in adolescents between the ages of 13 - 17 who have begun experiencing distressing yet subsyndromal psychosis or bipolar mood symptoms. Preliminary findings suggest feasibility and acceptability as well as initial efficacy in improving psychiatric symptoms, quality of life, and difficulties regulating emotions. We also present case studies from our open trial. A unified, cognitive-behavioral treatment for early presentations of SMI has important clinical and public health benefits, including streamlining treatment and providing broad skills that are applicable to a wide range of psychopathology.

Enrollment and Retention of Men and Women in Health Services Research and Development Trials

INTRODUCTION

Sex- and gender-specific science is essential to inform patient-centered, evidence-based care. Developing such evidence requires adequate inclusion of both women and men in trials. We sought to describe study participation of women and men in Department of Veterans Affairs Health Services Research and Development trials.

METHODS

We identified recent health services research trials from one Health Services Research and Development Center of Innovation and compared the participation of women and men from trial recruitment to study completion. We also calculated the participation to prevalence ratio (PPR) by sex for each trial.

RESULTS

We included eight trials that started recruitment between 2011 and 2014. Only one study purposefully attempted to boost the recruitment of women. Overall, the PPR for women ranged from 0.2 to 4.5, with seven studies having a PPR of greater than 1, indicating that women participated in these trials at proportions greater than their prevalence in the disease population within the Department of Veterans Affairs. The PPR for men ranged from 0.8 to 1.1. Retention was best with those studies that used administrative data for final outcomes assessment. No studies provided results stratified by sex or conducted analyses to explore treatment effect by sex.

CONCLUSIONS

At a single site, women participated in Health Services Research and Development trials at similar or greater rates to men without cross-study efforts to enrich the recruitment or retention of women. Adding strategic recruitment approaches could further boost the proportion of women in Department of Veterans Affairs trials and enable adequately powered sex-based analyses.

Exclusion criteria and generalizability in bipolar disorder treatment trials

Objective

The current paper reviews the English-language research on exclusion criteria in bipolar disorder treatment trials and discusses how study samples compare to the general bipolar patient population.

Methods

& Results: Across 8 identified studies of exclusion criteria and their impact, between 55% and 96% of people with bipolar disorder would be excluded from treatment research. The number of exclusion criteria varies across bipolar disorder treatment research, with one study estimate of a median of 7 criteria used across studies. The criteria that excluded the greatest number of potential participants were comorbid substance use disorder, suicidal risk, and comorbid medical conditions. Both studies that compared treatment responses among participants who met and did not meet exclusion criteria found no statistically significant differences.

Conclusions

Most potential participants are excluded from outcome research, which creates challenges for recruitment and limits generalizability of study findings. Common exclusionary practices lead to unrepresentative samples that limit generalizability and reduce the confidence of clinicians that findings can be translated to front-line practice with bipolar disorder patients.

Generalizability of Clinical Trial Results for Adolescent Major Depressive Disorder

BACKGROUND

Although there have been a number of clinical trials evaluating treatments for adolescents with major depressive disorder (MDD), the generalizability of those trials to samples of depressed adolescents who present for routine clinical care is unknown. Examining the generalizability of clinical trials of pharmacological and psychotherapy interventions for adolescent depression can help administrators and frontline practitioners determine the relevance of these studies for their patients and may also guide eligibility criteria for future clinical trials in this clinical population.

METHODS

Data on nationally representative adolescents were derived from the National Comorbidity Survey: Adolescent Supplement. To assess the generalizability of adolescent clinical trials for MDD, we applied a standard set of eligibility criteria representative of clinical trials to all adolescents in the National Comorbidity Survey: Adolescent Supplement with a Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition diagnosis of MDD (N = 592).

RESULTS

From the overall MDD sample, 61.9% would have been excluded from a typical pharmacological trial, whereas 42.2% would have been excluded from a psychotherapy trial. Among those who sought treatment (n = 412), the corresponding exclusion rates were 72.7% for a pharmacological trial and 52.2% for a psychotherapy trial. The criterion leading to the largest number of exclusions was "significant risk of suicide" in both pharmacological and psychotherapy trials.

CONCLUSIONS

Pharmacological and, to a lesser extent, psychotherapy clinical trials likely exclude most adolescents with MDD. Careful consideration should be given to balancing eligibility criteria and internal validity with applicability in routine clinical care while ensuring patient safety.

A trans-diagnostic review of anxiety disorder comorbidity and the impact of multiple exclusion criteria on studying clinical outcomes in anxiety disorders

Anxiety disorders are highly comorbid with each other and with other serious mental disorders. As our field progresses, we have the opportunity to pursue treatment study designs that consider these comorbidities. In this perspective review, we first characterized the prevalence of multiple anxiety disorder comorbidity by reanalyzing national survey data, then conducted an English-language PubMed search of studies analyzing the impact of exclusion criteria on treatment outcome data. In the prevalence data, 60% of people with an anxiety disorder had one or more additional anxiety or depression diagnosis. Because our commonly applied exclusion criteria focus on a single diagnosis and do not consider a multiple comorbidity profile, the impact of the criteria may be to exclude up to 92% of anxiety disorder treatment seekers. Moreover, the findings do not suggest a consistent relationship between the number of exclusion criteria and the effect size of treatment outcomes. Thus, future studies might consider a more trans-diagnostic rationale for determining exclusion criteria, one that is generalizable to real-world settings in which multiple diagnoses commonly co-occur. The findings also encourage a more systematic reporting of rationales for the choice of-and the implications of-each exclusion criterion.