Preference in random assignment: implications for the interpretation of randomized trials

Independent Supported Housing vs institutional housing rehabilitation settings for non-homeless individuals with severe mental illness - longitudinal results from an observational study

BACKGROUND

Most individuals with severe mental illness (SMI) strongly prefer independent living over living in an institution. Independent Supported Housing (ISH) provides housing rehabilitation for persons with SMI in their accommodations. However, most individuals who need housing rehabilitation live in institutional housing settings (housing rehabilitation as usual: HAU). We investigated which housing rehabilitation setting is effective on which variable in the long term to support service users to form an informed preference for either housing rehabilitation setting.

METHODS

We conducted a two-year longitudinal observational non-inferiority study to test the effectiveness of ISH in improving participants' social inclusion, quality of life, emotional social support, capabilities, symptom severity, functioning, service utilisation and costs. Participants were assessed at baseline and after six, twelve, and 24 months. Mixed effects models were computed to test between-group and within-group effects.

RESULTS

The study included 83 participants in ISH (n = 31) and HAU (n = 52) housing rehabilitation settings with a mean age of 36.2 years. Most participants were male (64%) and had a primary psychotic or schizophrenic (35%) or an affective diagnosis (24%). During the study, ISH participants significantly improved their quality of life (β = 0.54; 95% CI: 0.26 to 0.82), symptoms (β = -0.32; 95% CI: -0.60 to -0.03), and capabilities (β = 4.46; 95% CI: 0.14 to 8.77) and decreased psychiatric hospitalisations (p = 0.04). HAU participants improved their quality of life (β = 0.40; 95% CI: 0.12 to 0.69). Housing and rehabilitation support costs were almost half with ISH than with HAU.

CONCLUSION

ISH has been shown to be much less expensive than HAU and was associated with several improvements like reduced psychiatric hospitalisations and improved quality of life. Therefore, our findings strongly argue for a preference-driven provision of housing rehabilitation services and to end the institutionalisation of persons with SMI.

TRIAL REGISTRATION

The study was registered on December 04, 2018, at ClinicalTrials.gov (NCT03815604).

Exploring Factors Associated with Dietary Improvement Among Participants Who Completed a Randomized Controlled Trial for Physical Activity: Evidence of a Transfer Effect

BACKGROUND

This research aimed at understanding how changes in one behavior affect other behaviors, known as "transfer effect," could extend knowledge of shared constructs among compound health risk behaviors, and improve methods for promoting simultaneous behavior changes. The current study explored whether participants who completed a randomized controlled trial for physical activity (PA) improved their diet with no intervention in diet or nutrition.

METHODS

US adults (N = 283) were randomly assigned to 12 weeks of either exercise videogames, standard exercise, or an attention control. Secondary analyses were conducted to examine a possible transfer effect of the intervention on diet at end of intervention (EOT) and 6-month follow-up. Assessments of potential PA constructs (i.e., exercise enjoyment, self-efficacy) and demographics (e.g., age, gender) were conducted. PA, specifically moderate-to-vigorous levels of physical activity (MVPA), was measured using a self-report measure. Diet was measured using the Rate Your Plate dietary assessment.

RESULTS

Findings support that a randomization effect was associated with a higher likelihood of increasing MVPA (β = 30.00, 95% CI = 4.46, 64.46) and dietary improvement at EOT (β = 1.48, SE = .83, P = .01) and follow-up (β = 1.74, SE = .52, P = .02). At EOT, changes in diet were associated with higher PA enjoyment (β = 0.41, SE = 0.15, P = .01). This intervention effect on diet was moderated by gender, with women showing more dietary improvement than men (β =  - .78, SE = 13, P = .03). At 6 months, dietary improvement was associated with greater self-efficacy (β = .04, SE = .01, P = .01).

CONCLUSION

This study shows evidence of a transfer effect among two synergistic behaviors and expands understanding of factors that predict the occurrence of this type of behavior change.

Comparative effectiveness of group v. individual trauma-focused treatment for posttraumatic stress disorder in veterans

BACKGROUND

Cognitive processing therapy (CPT) and prolonged exposure (PE) delivered in an individual setting are efficacious and effective treatments for veterans with posttraumatic stress disorder (PTSD). Group CPT has been shown to be less efficacious than individual CPT, however, evidence regarding real-world effectiveness is limited.

METHODS

We conducted a retrospective, observational, comparative effectiveness study including veterans that received at least eight sessions of group CPT, individual CPT, or individual PE, and were discharged from PTSD residential treatment at the Department of Veterans Affairs between 1 October 2015, and 30 September 2020. PTSD symptom severity was assessed with the PTSD Checklist for DSM-5 (PCL-5) and treatments delivered in a group (CPT) or individual (CPT or PE) setting were compared at discharge and 4-month post-discharge follow-up.

RESULTS

Of 6735 veterans, 3888 [653 women (17%), median (IQR) age 45 (35-55) years] received individual and 2847 [206 women (7.2%), median (IQR) age 42 (34-54)] received group therapy. At discharge, improvement in PTSD severity was statistically greater among those treated individually (mean difference on the PCL-5, 2.55 (95% CI 1.61-3.49); p = <0.001]. However, the difference was smaller than the minimal clinically important difference of 7.9 points. The groups did not differ significantly at 4-month follow-up [mean difference on the PCL-5, 0.37 (95% CI -0.86 to 1.60); p = 0.551].

CONCLUSION

Group CPT was associated with a slightly smaller reduction of PTSD symptom severity than individual CPT or PE in veterans at the end of residential treatment. There were no differences at 4-month follow-up.

A scoping review on two-stage randomized preference trial in the field of mental health and addiction

BACKGROUND

Randomized Controlled Trial is the most rigorous study design to test the efficacy and effectiveness of an intervention. Patient preference may negatively affect patient performance and decrease the generalizability of a trial to clinical population. Patient preference trial have particular implications in the field of mental health and addiction since mental health interventions are generally complex, blinding of intervention is often difficult or impossible, patients may have strong preference, and outcome measures are often subjective patient self-report which may be greatly influenced if patient's preference did not match with the intervention received.

METHODS

In this review, we have surveyed the application of two-stage randomized preference trial with focus on studies in the field of mental health and addiction. The study selection followed the guideline provided by Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews.

RESULTS

Six two-stage randomized preference trials (ten publications) have been identified in the field of mental health field and addiction. In these trials, the pooled dropout rates were 18.3% for the preference arm, and 28.7% for the random arm, with a pooled RR of 0.70 (95% CI, 0.56-0.88; P = 0.010) indicating lower risk of dropout in the preference arm. The standardized preference effects varied widely from 0.07 to 0.57, and could be as large as the treatment effect in some of the trials.

CONCLUSION

This scoping review has shown that two-stage randomized preference trials are not as popular as expected in mental health research. The results indicated that two-stage randomized preference trials in mental health would be beneficial in retaining patients to expand the generalizability of the trial.

A mirror-image analysis of psychiatric hospitalisations among people with severe mental illness using Independent Supported Housing

BACKGROUND

Evidence on the effectiveness of Independent Supported Housing (ISH) for non-homeless people with severe mental illness primarily comes from observational cohort studies, which have high risk of bias due to confounding by time-invariant sample characteristics. The present study proposes an alternative study design known from pharmacology to overcome this bias and strengthen evidence.

METHODS

We conducted a retrospective mirror-image analysis with medical records of 144 ISH service users to assess the effectiveness of ISH in reducing the number and duration of hospitalisations. Outcomes occurring in equal periods before and during ISH utilisation were compared for every ISH user. Differences between the periods were tested with incidence rate ratios (IRR).

RESULTS

Included service users were on average 38.2 years old, female (54%) and predominately had an affective (28.5%) or a schizophrenic or psychotic (22.9%) disorder with ISH utilisation days ranging from 36-960. Fewer admissions (IRR = 0.41, 95%-CI 0.27-0.64) and fewer person-days hospitalised (IRR = 0.38, 95%-CI 0.35-0.41) were observed during ISH utilisation compared to prior to their ISH utilisation. While the reduction in psychiatric admissions may be somewhat confounded by time-variant characteristics, the substantial reduction in hospitalised bed-bays represents at least partially an intervention effect.

CONCLUSIONS

The mirror-image study design allowed for a cost-effective investigation of ISH effectiveness in reducing hospitalisation without confounding by time-invariant sample characteristics. We provide recommendations for the design's application and suggest further research with larger samples.

Independent Supported Housing for non-homeless individuals with severe mental illness: Comparison of two effectiveness studies using a randomised controlled and an observational study design

BACKGROUND

No randomised controlled study (RCT) on the effectiveness of Independent Supported Housing (ISH) vs. housing as usual (HAU) settings for non-homeless individuals with severe mental illness (SMI) has been conducted to date because of limited feasibility. Alternative designs, such as observational studies, might be suitable for providing adequate evidence if well conducted. To test this hypothesis, this article reports on a prospective, direct comparison of the designs of two parallel studies in this field.

METHODS

A two-centre, parallel-group non-inferiority effectiveness study was conducted at two locations in Switzerland using identical instruments and clinical hypotheses. One centre applied an RCT design and the other an observational study (OS) design with propensity score methods (ClinicalTrials.gov: NCT03815604). The comparability of the two study centres was investigated in terms of participants, procedures, and outcomes. The primary outcome was social inclusion and the secondary outcomes were quality of life and psychiatric symptoms.

RESULTS

The study included 141 participants (RCT: n = 58; OS: n = 83). Within one year, 27% study dropouts occurred (RCT: 34%; OS: 22%). A similar balance of sample characteristics was achieved in the RCT and the OS using propensity score methods (inverse probability of treatment weighting). After one year, ISH was non-inferior to the control condition regarding social inclusion (mean differences [95% CI]) in the RCT (6.28 [-0.08 to 13.35]) and the OS (2.24 [-2.30 to 6.77]) and showed no significant differences in quality of life (RCT: 0.12 [-0.52 to 0.75]; OS: 0.16 [-0.26 to 0.58]) and symptoms (RCT: -0.18 [-0.75 to 0.40]; OS: 0.21 [-0.17 to 0.60]) in both study centres. However, strong and persistent preferences for ISH in the RCT control group reduced participants' willingness to participate. Because of several limitations in the RCT, the results of the RCT and the OS are not comparable.

CONCLUSION

Participants were comparable in both study sites. However, there were significant problems in conducting the RCT because of strong preferences for ISH. The OS with propensity score methods provided results of more stable groups of participants and revealed balanced samples and valid outcome analysis. Our results do not support further investment in RCTs in this field.

Independent Housing and Support for non-homeless individuals with severe mental illness: randomised controlled trial vs. observational study - study protocol

BACKGROUND

Social inclusion is essential for an adequate rehabilitation process for people with serious mental illness (SMI). Various supported housing settings aim to promote housing competencies and social inclusion in service users. Nevertheless, there is a strong preference in service users for independent living. We aim to evaluate the effectiveness and efficiency of Independent Housing and Support (IHS) compared to institutionalised residential care settings and other treatment as usual conditions (RCS/TAU) in two cities in Switzerland.

METHODS

This is a prospective multi-centre, four-arm, non-inferiority cohort study investigating the effectiveness and efficiency of IHS and RCS/TAU for people with SMI. Effectiveness will be measured by a standardised measure of social inclusion as primary outcome as well as by measures of functioning and well-being. Efficiency will be analysed on the basis of service usage and costs associated with the different housing settings. Participants will be consecutively recruited and subsequently enrolled between April 2019 and December 2020 and assessed at baseline and after six, twelve and after 24 months. At one study site, 56 participants will be randomly assigned to one of the conditions; the other study site will be conducted as an observational study investigating 112 admitted participants.

DISCUSSION

While the UN Convention of the Rights of People with Disabilities aims to promote the opportunity to choose one's place of residence, the limited supply of alternative forms of housing does not guarantee genuine freedom of choice. Increased diversification and flexibility of housing support is essential. If IHS shows non-inferiority in terms of their effectiveness and efficiency, users should be allowed to choose their kind of housing support.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT03815604, December 04, 2019.

Housing First Improves Adherence to Antipsychotic Medication Among Formerly Homeless Adults With Schizophrenia: Results of a Randomized Controlled Trial

Adherence to antipsychotic medication is a significant challenge among homeless patients. No experimental trials have investigated the impact of Housing First on adherence among patients with schizophrenia. We investigated whether Housing First in congregate and scattered-site configurations resulted in superior adherence compared to usual care. Adult participants (n = 165) met criteria for homelessness, schizophrenia, and initiation of antipsychotic pharmacotherapy prior to recruitment to an unblinded, 3-arm randomized controlled trial in Vancouver, Canada. Randomization arms were: congregate Housing First (CHF) with on-site supports (including physician and pharmacy services); scattered-site Housing First (SHF) with Assertive Community Treatment; or treatment as usual (TAU) consisting of existing services. Participants were followed for an average of 2.6 years. Adherence to antipsychotic medication was measured using the medication possession ratio (MPR), and 1-way ANOVA was used to compare outcomes between the 3 conditions. Data were drawn from comprehensive pharmacy records. Prior to randomization, mean MPR among participants was very low (0.44–0.48). Mean MPR in the follow-up period was significantly different between study arms (P < .001) and approached the guideline threshold of 0.80 in SHF. Compared to TAU, antipsychotic adherence was significantly higher in SHF but not in CHF. The results demonstrate that further implementation of SHF is indicated among homeless people with schizophrenia, and that urgent action is needed to address very low levels of antipsychotic adherence in this population (trial registration: ISRCTN57595077).

Does Competitive Work Improve Quality of Life for Adults with Severe Mental Illness? Evidence from a Randomized Trial of Supported Employment

A randomized trial comparing a facility-based Clubhouse (N = 83) to a mobile Program of Assertive Community Treatment (PACT; N = 84) tested the widely held belief that competitive employment improves global quality of life for adults with severe mental illness. Random regression analyses showed that, over 24 months of study participation, competitively employed Clubhouse participants reported greater global quality of life improvement, particularly with the social and financial aspects of their lives, as well as greater self-esteem and service satisfaction, compared to competitively employed PACT participants. However, there was no overall association between global quality of life and competitive work, or work duration. Future research will determine whether these findings generalize to other certified Clubhouses or to other types of supported employment. Multi-site studies are needed to identify key mechanisms for quality of life improvement in certified Clubhouses, including the possibly essential role of Clubhouse employer consortiums for providing high-wage, socially integrated jobs.

A Conceptual Model to Promote the Retention of Women with Physical Disabilities in Research

Inadequate participant recruitment and retention practices can affect sample representativeness and thus the generalizability of research findings. Retention of research participants has been examined within the literature to some extent; however, there is no consensus on best practice in achieving acceptable results. Furthermore, there is a gap in understanding how to engage and retain women with physical disabilities (WPDs) in research. To address these oversights, we review (1) the significance of retention as a methodological concern, (2) factors that influence the involvement and retention of participants in research, including individual, population, and health-illness considerations, and (3) particular circumstances impacting the inclusion and retention of WPDs in research. On the basis of a review of the literature and our experience with the Healing Pathways randomized controlled trial (RCT), we present a conceptual model to guide culturally sensitive health research implementation with WPDs, and promote the engagement and retention of this group in RCTs and other forms of interventional health research.

Health promotion in individuals with mental disorders: a cluster preference randomized controlled trial

BACKGROUND

The existing literature on weight management interventions targeting physical activity and healthy eating in mental health care appears to provide only limited evidence. The aim of the study was to examine the effectiveness of a 10-week health promotion intervention, followed by a 6-month follow-up period in individuals with mental disorders living in sheltered housing in the Flanders region (Belgium).

METHODS

The study had a cluster preference randomized controlled design. Twenty-five sheltered housing organisations agreed to participate (16 in the intervention group, nine in the control group). In the intervention group, 225 individuals agreed to participate, while in the control group 99 individuals entered into the study. The main outcomes were changes in body weight, Body Mass Index, waist circumference and fat mass. Secondary outcomes consisted of changes in physical activity levels, eating habits, health-related quality of life and psychiatric symptom severity.

RESULTS

A significant difference was found between the intervention group and the control group regarding body weight (-0.35 vs. +0.22 kg; p=0.04), Body Mass Index (-0.12 vs. +0.08 kg/m2; p=0.04), waist circumference (-0.29 vs. + 0.55 cm; p<0.01), and fat mass (-0.99 vs. -0.12%; p<0.01). The decrease in these outcomes in the intervention group disappeared during the follow up period, except for fat mass. Within the intervention group, a larger decrease in the primary outcomes was found in the participants who completed the intervention. No significant differences between the two groups in changes in the secondary outcomes were found, except for the pedometer-determined steps/day. In the intervention group, the mean number of daily steps increased, while it decreased in the control group.

CONCLUSIONS

The study demonstrated that small significant improvements in the primary outcomes are possible in individuals with mental disorders. Integration of health promotion activities targeting physical activity and healthy eating into daily care are, however, necessary to maintain the promising results.

TRIAL REGISTRATION

This study is registered at ClinicalTrials.gov NCT 01336946.

Health promotion intervention in mental health care: design and baseline findings of a cluster preference randomized controlled trial

BACKGROUND

Growing attention is given to the effects of health promotion programs targeting physical activity and healthy eating in individuals with mental disorders. The design of evaluation studies of public health interventions poses several problems and the current literature appears to provide only limited evidence on the effectiveness of such programs. The aim of the study is to examine the effectiveness and cost-effectiveness of a health promotion intervention targeting physical activity and healthy eating in individuals with mental disorders living in sheltered housing. In this paper, the design of the study and baseline findings are described.

METHODS/DESIGN

The design consists of a cluster preference randomized controlled trial. All sheltered housing organisations in the Flanders region (Belgium) were asked if they were interested to participate in the study and if they were having a preference to serve as intervention or control group. Those without a preference were randomly assigned to the intervention or control group. Individuals in the intervention group receive a 10-week health promotion intervention above their treatment as usual. Outcome assessments occur at baseline, at 10 and at 36 weeks. The primary outcomes include body weight, Body Mass Index, waist circumference, and fat mass. Secondary outcomes consist of physical activity levels, eating habits, health-related quality of life and psychiatric symptom severity. Cost-effectiveness of the intervention will be examined by calculating the Cost-Effectiveness ratio and through economic modeling.Twenty-five sheltered housing organisations agreed to participate. On the individual level 324 patients were willing to participate, including 225 individuals in the intervention group and 99 individuals in the control group. At baseline, no statistical significant differences between the two groups were found for the primary outcome variables.

DISCUSSION

This is the first trial evaluating both the effectiveness and cost-effectiveness of a health promotion intervention targeting physical activity and healthy eating in mental health care using a cluster preference randomized controlled design. The baseline characteristics already demonstrate the unhealthy condition of the study population.

TRIAL REGISTRATION

This study is registered at clinicaltrials.gov - NCT 01336946.

Estimating the causal effect of randomization versus treatment preference in a doubly randomized preference trial

Although randomized studies have high internal validity, generalizability of the estimated causal effect from randomized clinical trials to real-world clinical or educational practice may be limited. We consider the implication of randomized assignment to treatment, as compared with choice of preferred treatment as it occurs in real-world conditions. Compliance, engagement, or motivation may be better with a preferred treatment, and this can complicate the generalizability of results from randomized trials. The doubly randomized preference trial (DRPT) is a hybrid randomized and nonrandomized design that allows for estimation of the causal effect of randomization versus treatment preference. In the DRPT, individuals are first randomized to either randomized assignment or choice assignment. Those in the randomized assignment group are then randomized to treatment or control, and those in the choice group receive their preference of treatment versus control. Using the potential outcomes framework, we apply the algebra of conditional independence to show how the DRPT can be used to derive an unbiased estimate of the causal effect of randomization versus preference for each of the treatment and comparison conditions. Also, we show how these results can be implemented using full matching on the propensity score. The methodology is illustrated with a DRPT of introductory psychology students who were randomized to randomized assignment or preference of mathematics versus vocabulary training. We found a small to moderate benefit of preference versus randomization with respect to the mathematics outcome for those who received mathematics training.

Viability of using employment rates from randomized trials as benchmarks for supported employment program performance

Cumulative employment rates published by randomized trials are based on each enrollee's pre-planned 18-24-months of study participation. By contrast, community programs typically report employment rates for clients active in services during a calendar quarter. Using data from three supported employment programs in randomized trials, we show that trial cumulative employment rates are about twice as large as quarterly employment rates for the same program. Therefore, we recommend that administrators, service networks, and mental health authorities begin to publish quarterly employment rates, and quarterly median earnings, to allow policymakers to set realistic performance expectations for supported employment programs.