Placebo control groups in randomized treatment trials: a statistician's perspective

Effectiveness of Videoconference-Delivered Cognitive Behavioral Therapy for Adults With Psychiatric Disorders: Systematic and Meta-Analytic Review

Background

Cognitive behavioral therapy (CBT) is the gold standard of psychotherapy for psychiatric disorders. However, the format of delivering CBT in person limits access to the intervention. The advancements in information and communication technology, especially the internet, present an opportunity for cognitive behavioral therapists to service patients or clients in remote areas through videoconferencing. Although many randomized controlled trials of videoconference-delivered cognitive behavioral therapy (VCBT) have already been conducted, the overall estimated effect size of VCBT for psychiatric disorders has not been examined by systematic reviews and meta-analyses.

Objective

This study attempts to evaluate the effectiveness of VCBT for psychiatric disorders through a systematic and meta-analytic review.

Methods

A systematic review and meta-analysis of studies in which VCBT was directly compared to control groups (such as treatment as usual, attention control, wait-list control, and other minimal supports) was carried out. To identify previous studies that meet our study objective, 2 independent reviewers undertook a systematic search through seven databases: MEDLINE (via PubMed), Web of Science, Science Direct, PsycINFO, CINAHL, LILACS, and SciELO. Other databases (ClinicalTrials.gov and Cochrane Central Resister of Controlled Trials) were also checked. All studies included in the review were assessed using the quality criteria of the Cochrane Collaboration. Statistical analysis was performed by using Cochrane Review Manager (RevMan, version 5.4.0). Standardized mean difference was used in major meta-analyses where a P value of .05 or less was the threshold for statistical significance. A heterogeneity test and the chi-square test were performed to assess the presence and extent of statistical heterogeneity with significance set at P<.10. Funnel plots were visually inspected to assess the risk of bias. Subgroup analyses were conducted for each disorder to estimate intervention effects.

Results

The systematic search resulted in 16 studies (total N=1745) that met the criteria for this study and were included in the review. There were 10 studies on depressive symptoms, 3 on chronic pain, 1 on generalized anxiety disorder, 1 on obsessive-compulsive disorder, and 1 on hypochondriasis. The quality and risk of bias was also assessed. Results showed a pooled effect size (Hedge g) post treatment of −0.49 (95% CI –0.68 to –0.29), indicating that VCBT is effective for clients with psychiatric disorders. Study quality did not affect outcomes.

Conclusions

While the overall results indicate the effectiveness of VCBT, there are still only a limited number of studies on specific psychiatric and somatic conditions. Therefore, more randomized controlled trials are needed to establish the effectiveness of VCBT for different disorders.

Trial Registration

International Prospective Register of Systematic Reviews (PROSPERO) CRD42021224832; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=224832

What have we learned about trial design from NIMH-funded pragmatic trials?

At the 2008 annual meeting of the American College of Neuropsychopharmacology (ACNP), a symposium was devoted to the following question: 'what have we learned about the design of pragmatic clinical trials (PCTs) from the recent costly long-term, large-scale trials of psychiatric treatments?' in order to inform the design of future trials. In all, 10 recommendations were generated placing emphasis on (1) appropriate conduct of pragmatic trials; (2) clinical, rather than, merely statistical significance; (3) sampling from the population clinicians are called upon to treat; (4) clinical outcomes of patients, rather than, on outcome measures; (5) use of stratification, controlling, or adjusting when necessary and not otherwise; (6) appropriate consideration of site differences in multisite studies; (7) encouragement of 'post hoc' exploration to generate (not test) hypotheses; (8) precise articulation of the treatment strategy to be tested and use of the corresponding appropriate design; (9) expanded opportunity for training of researchers and reviewers in RCT principles; and (10) greater emphasis on data sharing.

Preference in random assignment: implications for the interpretation of randomized trials

Random assignment to a preferred experimental condition can increase service engagement and enhance outcomes, while assignment to a less-preferred condition can discourage service receipt and limit outcome attainment. We examined randomized trials for one prominent psychiatric rehabilitation intervention, supported employment, to gauge how often assignment preference might have complicated the interpretation of findings. Condition descriptions, and greater early attrition from services-as-usual comparison conditions, suggest that many study enrollees favored assignment to new rapid-job-placement supported employment, but no study took this possibility into account. Reviews of trials in other service fields are needed to determine whether this design problem is widespread.

Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials

OBJECTIVE

Cognitive-behavioral therapy (CBT) is frequently used for various adult anxiety disorders, but there has been no systematic review of the efficacy of CBT in randomized placebo-controlled trials. The present study meta-analytically reviewed the efficacy of CBT versus placebo for adult anxiety disorders.

DATA SOURCES

We conducted a computerized search for treatment outcome studies of anxiety disorders from the first available date to March 1, 2007. We searched MEDLINE, PsycINFO, PubMed, Scopus, the Institute of Scientific Information, and Dissertation Abstracts International for the following terms: random*, cognitive behavior*therap*, cognitive therap*, behavior*therap*, GAD, generalized anxiety disorder, OCD, obsessive compulsive disorder, social phobia, social anxiety disorder, specific phobia, simple phobia, PTSD, post-traumatic stress disorder, and acute stress disorder. Furthermore, we examined reference lists from identified articles and asked international experts to identify eligible studies.

STUDY SELECTION

We included studies that randomly assigned adult patients between ages 18 and 65 years meeting DSM-III-R or DSM-IV criteria for an anxiety disorder to either CBT or placebo. Of 1165 studies that were initially identified, 27 met all inclusion criteria.

DATA EXTRACTION

The 2 authors independently identified the eligible studies and selected for each study the continuous measures of anxiety severity. Dichotomous measures reflecting treatment response and continuous measures of depression severity were also collected. Data were extracted separately for completer (25 studies for continuous measures and 21 studies for response rates) and intent-to-treat (ITT) analyses (6 studies for continuous measures and 8 studies for response rates).

DATA SYNTHESIS

There were no significant differences in attrition rates between CBT and placebo. Random-effects models of completer samples yielded a pooled effect size (Hedges' g) of 0.73 (95% CI = 0.88 to 1.65) for continuous anxiety severity measures and 0.45 (95% CI = 0.25 to 0.65) for depressive symptom severity measures. The pooled odds ratio for completer treatment response rates was 4.06 (95% CI = 2.78 to 5.92). The strongest effect sizes were observed in obsessive-compulsive disorder and acute stress disorder, and the weakest effect size was found in panic disorder. The advantage of CBT over placebo did not depend on placebo modality, number of sessions, or study year.

CONCLUSIONS

Our review of randomized placebo-controlled trials indicates that CBT is efficacious for adult anxiety disorders. There is, however, considerable room for improvement. Also, more studies need to include ITT analyses in the future.

Feasibility of reducing the duration of placebo-controlled trials in schizophrenia research

Use of placebo-controlled trials in medical and psychiatric research has been controversial, although a consensus is emerging about conditions under which placebo-controlled trials are ethical. In schizophrenia research, the paradigm of slow onset of antipsychotic effects has led to a model in which placebo-controlled trials of 6-8 weeks duration have been used to demonstrate efficacy. Recent evidence that the largest symptom reductions are typically seen in the first weeks of treatment suggests that shorter placebo-controlled studies to demonstrate antipsychotic efficacy are possible. In a pilot study of the feasibility of shortening placebo-controlled studies, we reanalyzed data from placebo-controlled registry trials of olanzapine and risperidone and found that trials as short as 4 weeks could have similar power to longer term 6-8 week studies, given the estimated time course of treatment effects. Although fuller evaluation is required, the results suggest future antipsychotic trials could be shortened from 6-8 weeks to 3-4 weeks with a relatively low increase in sample size requirements. Shortening placebo-controlled trials would reduce patient burden and ethical objections to prolonged administration of placebo and reduce potential bias due to high dropout rates in longer clinical trials.

Emerging empirical evidence on the ethics of schizophrenia research

Many challenging ethical questions come with the scientific efforts to understand the nature and treatment of schizophrenia. The empirical study of ethical aspects of schizophrenia research has sought to clarify and resolve many of these questions. In this article we provide an overview of the existing data-based literature on schizophrenia research ethics and outline directions for future inquiry. We examine 5 broad categories of inquiry into the ethics of schizophrenia research: (1) Scientific designs (eg, placebo-controlled studies and medication-free intervals, prodromal and high-risk research, and genetics research); (2) informed consent and decision-making capacity, including assessment of decisional abilities, as well as intervention studies; (3) understanding and perceptions of risk and benefit (including the therapeutic misconception); (4) influences on research participation (including voluntarism, altruism, and other motivations); and (5) key participant safeguards, such as protocol review and participant advocates. We discuss how empirical work in each of these areas answers certain questions and raises new ones. Finally, we highlight important gaps in our understanding of ethically relevant aspects of schizophrenia research and offer a specific research agenda for empirical ethics.

Ethische Implikationen plazebokontrollierter Prüfungen von Psychopharmaka

The "note of clarification" of the World Medical Association to Article 29 of the Fifth Revision of the Declaration of Helsinki (Edinburgh 2000) intensified the controversial discussion of how methodologically compelling and ethically acceptable pure placebo controls are in clinical trials of new drugs if an effective standard therapy ("active control") is available. Pros and cons from this discussion are exposed and examined with regard to "minor" psychic disorders, depressions, and schizophrenia. Criteria are developed to facilitate the evaluation of each trial application. These criteria also seem to be necessary because normative authorities, which are responsible for licensing drugs such as FDA and EMEA, and legally relevant regulations such as the additional protocol for biomedical research of the European Council do not agree with regard to placebo controls. It is concluded that pure placebo-controlled clinical trials of new psychotropic drugs may be acceptable in minor psychic disorders, but in general -- at least in a pure form -- not in major disorders such as schizophrenia and major depression.

Impact of referral source and study applicants' preference for randomly assigned service on research enrollment, service engagement, and evaluative outcomes

OBJECTIVE

The inability to blind research participants to their experimental conditions is the Achilles' heel of mental health services research. When one experimental condition receives more disappointed participants, or more satisfied participants, research findings can be biased in spite of random assignment. The authors explored the potential for research participants' preference for one experimental program over another to compromise the generalizability and validity of randomized controlled service evaluations as well as cross-study comparisons.

METHOD

Three Cox regression analyses measured the impact of applicants' service assignment preference on research project enrollment, engagement in assigned services, and a service-related outcome, competitive employment.

RESULTS

A stated service preference, referral by an agency with a low level of continuity in outpatient care, and willingness to switch from current services were significant positive predictors of research enrollment. Match to service assignment preference was a significant positive predictor of service engagement, and mismatch to assignment preference was a significant negative predictor of both service engagement and employment outcome.

CONCLUSIONS

Referral source type and service assignment preference should be routinely measured and statistically controlled for in all studies of mental health service effectiveness to provide a sound empirical base for evidence-based practice.

Reducing exposure of clinical research subjects to placebo treatments

The ethics of pill placebo and placebo psychotherapy conditions in clinical research are controversial. Even when not life threatening, mental disorders dramatically diminish the quality of life. Pill-placebo conditions in drug treatment research have been justified on the grounds that a placebo versus standard drug comparison is necessary to test the quality of the study, viz., the assay sensitivity method. The assay sensitivity method of judging study quality, however, results in misclassification of the quality of some studies, leading to bias in effect size estimation in the context of meta-analyses. This bias is of particular concern in relation to studies comparing psychotherapies to psychotropic drugs, which are conducted outside of the Food and Drug Administration (FDA) context. In cases in which control conditions may be justified on grounds other than as essential elements of an assay sensitivity test, statistical methods to reduce the number of study participants exposed to placebo should be strongly considered. Of the methods available, group sequential methods are the most widely used. Group sequential methods involve successive looks at accumulating data, with rules for terminating a trial (or an arm of a trial) early if results are strong enough.

Placebo-free designs for evaluating new mental health treatments: the use of adaptive treatment strategies

The dominant pre-marketing clinical trial in psychopharmacology is a non-equivalence design that randomizes patients to one of three treatments: an accepted standard, the innovation (new drug), or placebo, with the main efficacy comparison being innovation vs placebo. The reasons behind the choice of placebo control in new drug development include anticipated small effect size for active-controlled comparisons and the sufficiency of demonstrated treatment effect (new drug vs placebo) for regulatory approval. These reasons have led to great reliance on placebo control in drug evaluation studies, despite the ethical controversy over the use of placebo when there are known effective standard treatments. While the use of placebo controls has been widely debated, a less considered aspect of the usual placebo-controlled non-equivalence design is the disparity between the decisions that it supports and those that pervade clinical practice. We propose an alternative approach that randomizes one group of patients to an adaptive treatment strategy that exemplifies the adaptive nature of clinical decision-making in the treatment of ongoing mental health disorders. The basic idea is to compare the adaptive strategy, which uses a patient's outcomes to date to determine when to switch from an initial treatment (e.g. an accepted standard) to an alternative (e.g. the new) treatment, to fixed trials of either treatment option. We state the conditions under which the adaptive treatment RCT is attractive to implement and the requirements for doing so.

Treatment research in bipolar disorder: issues and recommendations

Bipolar (manic-depressive) disorder is one of the most common of the severe mental illnesses. Officially recognised forms comprise type I (with mania), type II (with hypomania), cyclothymia and a rapid-cycling subtype. International lifetime prevalence estimates are 1 to 5% of the general population, and bipolar disorder accounts disproportionately for idiopathic psychoses. Psychiatric and substance-abuse comorbidities are common complications, and mortality rates are increased as a result of high suicidal risks, accidents, complications of substance abuse and increased fatality of stress-sensitive medical illnesses. Complex and labile symptomatic presentations, a tendency for patients to deny illness and reject treatment, and diagnostic heterogeneity severely complicate the design, conduct and interpretation of experimental treatment trials in bipolar disorder. Progress in the short-term treatment of mania with certain antiepileptic drugs and atypical antipsychotic agents has advanced greatly in recent years; however, long-term treatment trials other than with lithium remain rare, as are studies of type II disorder, bipolar depression and mixed states, and there is limited information on treatment effectiveness against comorbidity, dysfunction and mortality. There is a growing realisation that bipolar disorder represents a major, largely unmet, international public health challenge and that innovative methods for carrying out reliable and generalisable long-term pharmacological treatment trials, alone and in combination with cost-effective psychosocial and rehabilitative interventions, are urgently required.

Catching up on health outcomes: the Texas Medication Algorithm Project

OBJECTIVE

To develop a statistic measuring the impact of algorithm-driven disease management programs on outcomes for patients with chronic mental illness that allowed for treatment-as-usual controls to "catch up" to early gains of treated patients.

DATA SOURCES/STUDY SETTING

Statistical power was estimated from simulated samples representing effect sizes that grew, remained constant, or declined following an initial improvement. Estimates were based on the Texas Medication Algorithm Project on adult patients (age > or = 18) with bipolar disorder (n = 267) who received care between 1998 and 2000 at 1 of 11 clinics across Texas.

STUDY DESIGN

Study patients were assessed at baseline and three-month follow-up for a minimum of one year. Program tracks were assigned by clinic.

DATA COLLECTION/EXTRACTION METHODS

Hierarchical linear modeling was modified to account for declining-effects. Outcomes were based on 30-item Inventory for Depression Symptomatology-Clinician Version.

PRINCIPAL FINDINGS

Declining-effect analyses had significantly greater power detecting program differences than traditional growth models in constant and declining-effects cases. Bipolar patients with severe depressive symptoms in an algorithm-driven, disease management program reported fewer symptoms after three months, with treatment-as-usual controls "catching up" within one year.

CONCLUSIONS

In addition to psychometric properties, data collection design, and power, investigators should consider how outcomes unfold over time when selecting an appropriate statistic to evaluate service interventions. Declining-effect analyses may be applicable to a wide range of treatment and intervention trials.

The placebo response in social phobia

The placebo response forms a growing problem in randomized, placebo-controlled clinical trials in psychiatry. Research into the placebo response is on the increase, but remains very limited in relation to social phobia. Together with the dropout rate, the placebo effect is an important factor limiting the discriminative properties of any study. In this study, we reviewed 15 placebo-controlled studies in social phobia, focussing on patients and study characteristics. In social phobia, the placebo effect has turned out to be moderately large and has shown no increase over the past decade. Placebo response was highest in large, multicentred trials and was independent of study duration. No validation for a placebo run-in was found. Taking into account both response to placebo and active drug, as well as dropout rate, the most discriminative results are probably to be expected in a sample of patients who are moderately to severely impaired. More research in the field of the placebo response is needed.