Assessing Statistical Methods for Causal Inference in Observational Data

Independent Supported Housing for non-homeless individuals with severe mental illness: Comparison of two effectiveness studies using a randomised controlled and an observational study design

BACKGROUND

No randomised controlled study (RCT) on the effectiveness of Independent Supported Housing (ISH) vs. housing as usual (HAU) settings for non-homeless individuals with severe mental illness (SMI) has been conducted to date because of limited feasibility. Alternative designs, such as observational studies, might be suitable for providing adequate evidence if well conducted. To test this hypothesis, this article reports on a prospective, direct comparison of the designs of two parallel studies in this field.

METHODS

A two-centre, parallel-group non-inferiority effectiveness study was conducted at two locations in Switzerland using identical instruments and clinical hypotheses. One centre applied an RCT design and the other an observational study (OS) design with propensity score methods (ClinicalTrials.gov: NCT03815604). The comparability of the two study centres was investigated in terms of participants, procedures, and outcomes. The primary outcome was social inclusion and the secondary outcomes were quality of life and psychiatric symptoms.

RESULTS

The study included 141 participants (RCT: n = 58; OS: n = 83). Within one year, 27% study dropouts occurred (RCT: 34%; OS: 22%). A similar balance of sample characteristics was achieved in the RCT and the OS using propensity score methods (inverse probability of treatment weighting). After one year, ISH was non-inferior to the control condition regarding social inclusion (mean differences [95% CI]) in the RCT (6.28 [-0.08 to 13.35]) and the OS (2.24 [-2.30 to 6.77]) and showed no significant differences in quality of life (RCT: 0.12 [-0.52 to 0.75]; OS: 0.16 [-0.26 to 0.58]) and symptoms (RCT: -0.18 [-0.75 to 0.40]; OS: 0.21 [-0.17 to 0.60]) in both study centres. However, strong and persistent preferences for ISH in the RCT control group reduced participants' willingness to participate. Because of several limitations in the RCT, the results of the RCT and the OS are not comparable.

CONCLUSION

Participants were comparable in both study sites. However, there were significant problems in conducting the RCT because of strong preferences for ISH. The OS with propensity score methods provided results of more stable groups of participants and revealed balanced samples and valid outcome analysis. Our results do not support further investment in RCTs in this field.