INdividual Vocational and Educational Support Trial (INVEST) for young people with borderline personality disorder: study protocol for a randomised controlled trial

Background

The clinical onset of borderline personality disorder (BPD) usually occurs in young people (aged 12–25 years) and commonly leads to difficulty achieving and maintaining vocational (education and/or employment) engagement. While current psychosocial interventions lead to improvements in psychopathology, they have little effect upon functioning. Individual Placement and Support (IPS) is a client-driven model that assists individuals with severe mental illness to engage with education and/or employment appropriate to their personal goals, and that provides ongoing support to maintain this engagement. The objective of the INdividual Vocational and Educational Support Trial (INVEST) is to evaluate the effectiveness of adding IPS to an evidence-based early intervention programme for BPD, with the aim of improving vocational outcomes.

Methods/design

INVEST is a single-blind, parallel-groups, randomised controlled trial (RCT). The randomisation is stratified by gender and age and uses random permuted blocks. The interventions are 39 weeks of either IPS, or ‘usual vocational services’ (UVS). Participants will comprise 108 help-seeking young people (aged 15–25 years) with three or more DSM-5 BPD features and a desire to study or work, recruited from the Helping Young People Early (HYPE) early intervention programme for BPD at Orygen, in Melbourne, Australia. All participants will receive the HYPE intervention. After baseline assessment, staff who are blind to the intervention group allocation will conduct assessments at 13, 26, 39 and 52 weeks. At the 52-week primary endpoint, the primary outcome is the number of days in mainstream education/employment since baseline. Secondary outcomes include the cost-effectiveness of the intervention, quality of life, and BPD severity.

Discussion

Current treatments for BPD have little impact on vocational outcomes and enduring functional impairment is prevalent among this patient group. IPS is a targeted functional intervention, which has proven effective in improving vocational outcomes for adults and young people with psychotic disorders. This trial will investigate whether IPS is effective for improving vocational (employment and educational) outcomes among young people with subthreshold or full-syndrome BPD.

Trial registration

Australian New Zealand Clinical Trials Registry, ID: ACTRN12619001220156. 13 September 2019.

Long-Term Follow-Up of a Randomized Trial of Supported Employment for SSDI Beneficiaries With Mental Illness

OBJECTIVE

In this study, the authors assessed the long-term impact of the Mental Health Treatment Study (MHTS), a randomized controlled trial testing the effects of providing 2 years of employment services based on the evidence-based individualized placement and support model to Social Security Disability Insurance (SSDI) recipients with serious mental illness. Treatment recipients also received systematic medication management, supplemental health care supports, and short-term relief from medical continuing disability review by the Social Security Administration (SSA).

METHODS

MHTS site data for 2,160 participants were linked to SSA administrative data from 2011 to 2015, 1 to 5 years after the original study concluded. Univariate and multivariate models were used to assess the MHTS effects on employment, earnings, and disability benefit suspension-termination up to 7 years after services ended.

RESULTS

The analyses showed that the treatment group was more likely than the control group to work, and average earnings among the treatment group increased more over time than earnings among the control group. Disability benefit suspension/termination did not differ between groups.

CONCLUSIONS

Providing the demonstration's package of services and support to SSDI beneficiaries with psychiatric disabilities for up to 2 years may have a long-term impact on employment and earnings. Under the SSDI program as currently structured, however, even after receiving 2 years of evidence-based supported employment and high-quality mental health services, SSDI beneficiaries with psychiatric conditions are unlikely to achieve economic independence within 5 years.

Commentary on special issue on individual placement and support (IPS) international

OBJECTIVE

In this commentary, we identify major themes found in this set of 11 articles, which examine the history of individual placement and support (IPS) adoption within each nation.

METHOD

We summarize major barriers and strategies to achieve to IPS adoption and implementation and briefly discuss IPS research conducted in these countries.

FINDINGS

Faced with many barriers, countries are developing strategies to promote IPS implementation and dissemination. IPS research has consistently found superior employment outcomes for people receiving IPS compared to usual services. Researchers are evaluating the effectiveness of IPS for new populations.

CONCLUSIONS AND IMPLICATIONS FOR PRACTICE

IPS is expanding worldwide, aided by strong leadership, national directives, and the International IPS Learning Community. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Psychometric Properties of the General Organizational Index (GOI): A Measure of Individualization and Quality Improvement to Complement Program Fidelity

To assess the implementation of effective practices, mental health programs need standardized measures. The General Organizational Index (GOI), although widely used for this purpose, has received minimal psychometric research. For this study, we assessed psychometric properties of the GOI scale administered four times over 18 months during the implementation of a new program in 11 sites. The GOI scale demonstrated high levels of interrater reliability (.97), agreement between assessors on item ratings (86% overall), internal consistency (.77–.80 at three time points), sensitivity to change, and feasibility. We conclude that the GOI scale has acceptable psychometric properties, and its use may enhance implementation and research on evidence-based mental health practices.

Trial registration: REK2015/2169. ClinicalTrials.gov Identifier: NCT03271242

Psychometric Properties of a Fidelity Scale for Illness Management and Recovery

This study examined the psychometric properties and feasibility of the Illness Management and Recovery (IMR) Fidelity scale. Despite widespread use of the scale, the psychometric properties have received limited attention. Trained fidelity assessors conducted assessments four times over 18 months at 11 sites implementing IMR. The IMR Fidelity scale showed excellent interrater reliability (.99), interrater item agreement (94%), internal consistency (.91–.95 at three time points), and sensitivity to change. Frequency distributions generally showed that item ratings included the entire range. The IMR Fidelity scale has excellent psychometric properties and should be used to evaluate and guide the implementation of IMR.

Trial registration: ClinicalTrials.gov Identifier: NCT03271242.

Assessing the Fidelity of Evidence-Based Practices: History and Current Status of a Standardized Measurement Methodology

Evidence-based practices are effective only when implemented faithfully. This paper explicates the history, standardization, and methods for developing and validating measures of fidelity. We overviewed the past 20 years of developing fidelity measures, summarized standardization of the development procedures, and described needed psychometric assessments. Fidelity assessment has become the sine qua non of implementation, technical assistance, and research on evidence-based practices. Researchers have established standardized procedures for scale development and psychometric testing. Widescale use of fidelity measurement remains challenging. The implementation of evidence-based practice and the development and validation of fidelity measures are interdependent. International improvements of mental health care will require attention to both.

Measuring implementation: development of the implementation process assessment tool (IPAT)

BACKGROUND

Implementation science comprises a large set of theories suggesting interacting factors at different organisational levels. Development of literature syntheses and frameworks for implementation have contributed to comprehensive descriptions of implementation. However, corresponding instruments for measuring these comprehensive descriptions are currently lacking. The present study aimed to develop an instrument measuring care providers' perceptions of an implementation effort, and to explore the instrument's psychometric properties.

METHODS

Based on existing implementation literature, a questionnaire was designed with items on individual and team factors and on stages of change in an implementation process. The instrument was tested in a Norwegian study on implementation of evidence based practices for psychosis. Item analysis, factor structure, and internal consistency at baseline were examined.

RESULTS

The 27-item Implementation Process Assessment Tool (IPAT) revealed large variation between mean score of the items. The total scale scores were widely dispersed across respondents. Internal consistency for the total scale was high (Cronbach's alpha: .962), and all but one item contributed positively to the construct. The results indicated four underlying constructs: individual stages for behavioural change, individual activities and perceived support, collective readiness and support, and individual perceptions of the intervention.

CONCLUSIONS

The IPAT appears to be a feasible instrument for investigating the implementation process from the perspective of those making the change. It can enable examination of the relative importance of factors thought to be essential for implementation outcomes. It may also provide ongoing feedback for leaders tailoring support for teams to improve implementation. However, further research is needed to detect the instrument's properties later in the implementation process and in different contexts.

TRIAL REGISTRATION

ClinicalTrials.gov code NCT03271242 (retrospective registered September 5, 2017).

A virtual learning collaborative to implement health promotion in routine mental health settings: Protocol for a cluster randomized trial

BACKGROUND

Despite widespread use of learning collaboratives in health care, few randomized trials have evaluated their effectiveness. The primary aim of this cluster randomized implementation trial is to evaluate the effectiveness of a virtual learning collaborative (VLC) in the implementation of a lifestyle intervention for persons with serious mental illness (SMI) in routine mental health settings, compared to standard individual technical assistance.

METHODS

Forty-eight mental health provider organizations from across the United States will be recruited to participate in the trial. The evidence-based practice to be implemented is the InSHAPE health promotion intervention for persons with SMI. Sites will be stratified by size and randomized to receive an 18-month intensive group-based VLC with monthly learning sessions or individual technical assistance with four scheduled conference calls over 18 months. Sites will be enrolled in three blocks of 16 sites each. The primary outcomes are InSHAPE program participation and fidelity, and participant weight loss; secondary outcomes are program operation, program uptake, participant health behaviors of physical activity and nutrition, organizational change, and program sustainment. Implementation outcomes are measured at 3, 6, 12, 18, and 24 months after the program start-up. Participant-level outcomes are measured at fixed intervals every 3 months after each participant enrolls in the study.

DISCUSSION

This study will determine whether VLCs are an effective implementation strategy among resource-limited providers when the new practice necessitates a shift in mission, scope of practice, type of services delivered, and new financing.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT03891368 Registered 25 March 2019, retrospectively registered. https://clinicaltrials.gov/ct2/show/NCT03891368?term=NCT03891368&rank=1.

Individual placement and support for young adults at risk of early work disability (the SEED trial). A randomized controlled trial

Objectives Individual placement and support (IPS) is an effective approach for helping people with severe mental illness gain employment. This study aimed to investigate if IPS can be effectively repurposed to support young adults at risk of early work disability due to various social and health related problems. Methods A randomized controlled trial including 96 young adults (18‒29 years; 68% men) was conducted in Norway. Participants were not in employment, education, or training, received temporary benefits due to social or health-related problems, and were eligible for traditional vocational rehabilitation (TVR). Participants were randomized to IPS (N=50) or TVR (N=46). Self-reported data were collected at baseline and at 6- and 12-months follow-up. The primary outcome was obtaining any paid employment in the competitive labor market during follow-up. Secondary outcomes were physical and mental health, well-being, coping, alcohol consumption, and drug use. Results Significantly more IPS participants obtained competitive employment compared to TVR participants during 12-months follow-up (48% versus 8%; odds ratio 10.39, 95% confidence interval 2.79‒38.68). The IPS group reported significantly better outcomes than the TVR group in subjective health complaints, helplessness, and hopelessness. In post hoc analyses adjusted for baseline and missing data, the IPS group reported significantly better outcomes on these measures in addition to level of disability, optimism about future well-being, and drug use. Conclusions IPS is effective for young adults at risk of early work disability. IPS was superior to TVR in increasing competitive employment and promoted improvements in some non-vocational outcomes. IPS services should be offered to improve employment rates in this vulnerable group.

Expanding Individual Placement and Support to Populations With Conditions and Disorders Other Than Serious Mental Illness

OBJECTIVE

A systematic review of studies of individual placement and support (IPS) for populations other than those with serious mental illness was conducted.

METHODS

The authors searched three electronic databases (PubMed, Web of Science, and Scopus) for studies of IPS and modified IPS. Eligibility criteria for the systematic review included randomized controlled trials with prospective data collection on competitive employment rate and at least 10 study participants from a well-defined population other than people with serious mental illness. Results were compiled for competitive employment rates, IPS fidelity, and other outcomes.

RESULTS

Three clinical groups other than people with serious mental illness have been studied: people with psychiatric disorders other than serious mental illness, people with substance use disorders, and people with musculoskeletal or neurological disorders. Nine controlled trials with a total of 2,902 participants included six trials with people who had psychiatric disorders other than serious mental illness, two with people who had substance use disorders, and one with people who had spinal cord injuries. In eight studies, results for competitive employment rates significantly favored IPS. Meta-analysis yielded an overall weighted odds ratio of 2.23 (95% confidence interval=1.53-3.24, p<.001). Findings for other employment outcomes also favored IPS, but findings on symptom reduction and quality of life were inconsistent. The strongest (and only replicated) findings were for veterans with posttraumatic stress disorder (PTSD). Methodological limitations included small samples, major modifications to IPS fidelity, and short follow-up periods.

CONCLUSIONS

IPS, often with modifications, is a promising employment intervention for several populations in addition to people with serious mental illnesses. The strongest evidence pertains to veterans with PTSD. IPS should be offered to these veterans. Research on other populations, including people with anxiety, depression, substance use disorder, musculoskeletal or neurological conditions, or pain syndromes, needs development, amplification, and replication.

Introducing Individual Placement and Support (IPS) supported employment in Japan

Individual Placement and Support is a standardized model of supported employment, developed initially for people with serious mental disorders and now applied to people with a range of disabilities. More than two dozen randomized controlled trials around the world show that the approach helps a majority of participants to succeed in competitive employment. Individual Placement and Support is spreading rapidly across the USA and in many other high-income countries, with facilitation by an international learning community.

Comparing predictors of employment in Individual Placement and Support: A longitudinal analysis

Individual Placement and Support (IPS) is an evidence-based model of supported employment for people with serious mental illness. We assessed the effects and relative contributions of predictors of employment among IPS recipients using measures of baseline client characteristics, local economic context, and IPS fidelity. A recent work history, less time on the Social Security rolls, greater cognitive functioning, and a lower local unemployment rate were associated with greater probability of employment. The ability of the model to discriminate between outcomes was limited, and substantial improvements in our understanding of IPS employment outcomes will require the study of novel client, environmental, and IPS implementation factors.

Economic, Labor, and Regulatory Moderators of the Effect of Individual Placement and Support Among People With Severe Mental Illness: A Systematic Review and Meta-analysis

As Individual Placement and Support (IPS) has become the international standard for vocational rehabilitation of adults with serious mental illness, researchers must consider the relationship between IPS and local environments. This meta-analysis used mixed-effects meta-regressions to assess the impact of site-level moderators on the likelihood that IPS recipients, compared with recipients of alternative vocational services, achieved competitive employment. Potential moderators included change in gross domestic product (GDP), local unemployment and unionization rates, and indices describing employment protection regulations, level of disability benefits compensation, and efforts to integrate people with disabilities into the workforce. Regulatory moderators represent facilitators and barriers to employment that may reinforce or detract from the effectiveness of IPS. Across 30 sites drawn from 21 randomized controlled trials in 12 countries (33% in the United States), IPS recipients were 2.31 (95% CI 1.99-2.69) times more likely to find competitive employment than recipients of alternative vocational rehabilitation services. The significant competitive-employment rate advantage of IPS over control services increased in the presence of weaker employment protection legislation and integration efforts, and less generous disability benefits. Policy makers should recognize and account for the fact that labor and disability regulations can create an arrangement of incentives that reduces the relative efficacy of supported employment.

Barriers to employment for people with severe mental illness and criminal justice involvement

Background: People with severe mental illness (SMI) and criminal justice involvement (CJI) are able to achieve competitive employment when provided with evidence-based services, but outcomes are modest compared to studies of SMI in general.Aims: To investigate barriers to employment facing people with SMI and CJI receiving employment services.Method: Employment service providers assessed top three barriers to employment for 87 people with SMI and CJI enrolled in a randomized controlled trial comparing a job club program to supported employment. Main barriers were identified and differences between programs were analyzed. Associations between barriers and client background characteristics were investigated.Results: The most common barriers were failure to engage and disengagement from services, followed by substance abuse. Staff from the two employment programs reported similar barriers.Conclusions: Engagement problems were the single most important barrier to employment across programs. Surprisingly, criminal history was rarely mentioned. This may be explained by barriers arising earlier in the process, before achieving employer contact. The results call for recognition and awareness of the importance of motivational issues as well as factors that may inhibit engagement in services, highlighting possible augmentations to evidence-based employment services that may be necessary in the rehabilitation of this patient group.

Barriers and Facilitators to Sustainment of an Evidence-Based Supported Employment Program

Large-scale initiatives to expand evidence-based practices are often poorly implemented and rarely endure. The purpose of this study was to identify the perceived barriers and facilitators to sustainment of an evidence-based supported employment program, Individual Placement and Support (IPS). Within a 2-year prospective study of sustainment among 129 IPS programs in 13 states participating in a national learning community, we interviewed IPS team leaders and coded their responses to semi-structured interviews using a conceptual framework adapted from another large-scale implementation study. Leaders in 122 agencies (95%) that sustained their IPS programs identified funding, prioritization, and workforce characteristics as both key facilitators and barriers. Additional key factors were lack of local community supports as a barrier and leadership and structured workflow as facilitators. Within the IPS learning community, team leaders attributed the sustainment of their program to funding, prioritization, workforce, agency leadership, and structured workflow. The actions of the learning community's leadership, state governments, and local programs together may have contributed to the high sustainment rate.

Development of a measure of model fidelity for mental health Crisis Resolution Teams

BACKGROUND

Crisis Resolution Teams (CRTs) provide short-term intensive home treatment to people experiencing mental health crisis. Trial evidence suggests CRTs can be effective at reducing hospital admissions and increasing satisfaction with acute care. When scaled up to national level however, CRT implementation and outcomes have been variable. We aimed to develop and test a fidelity scale to assess adherence to a model of best practice for CRTs, based on best available evidence.

METHODS

A concept mapping process was used to develop a CRT fidelity scale. Participants (n = 68) from a range of stakeholder groups prioritised and grouped statements (n = 72) about important components of the CRT model, generated from a literature review, national survey and qualitative interviews. These data were analysed using Ariadne software and the resultant cluster solution informed item selection for a CRT fidelity scale. Operational criteria and scoring anchor points were developed for each item. The CORE CRT fidelity scale was then piloted in 75 CRTs in the UK to assess the range of scores achieved and feasibility for use in a 1-day fidelity review process. Trained reviewers (n = 16) rated CRT service fidelity in a vignette exercise to test the scale's inter-rater reliability.

RESULTS

There were high levels of agreement within and between stakeholder groups regarding the most important components of the CRT model. A 39-item measure of CRT model fidelity was developed. Piloting indicated that the scale was feasible for use to assess CRT model fidelity and had good face validity. The wide range of item scores and total scores across CRT services in the pilot demonstrate the measure can distinguish lower and higher fidelity services. Moderately good inter-rater reliability was found, with an estimated correlation between individual ratings of 0.65 (95% CI: 0.54 to 0.76).

CONCLUSIONS

The CORE CRT Fidelity Scale has been developed through a rigorous and systematic process. Promising initial testing indicates its value in assessing adherence to a model of CRT best practice and to support service improvement monitoring and planning. Further research is required to establish its psychometric properties and international applicability.

Sustaining and Expanding Evidence-Based Supported Employment: The Role of State Leaders Participating in a Learning Community

State leaders often promote implementation of evidence-based practices but have difficulty sustaining and expanding them over time. This paper examines the activities of leaders in 13 states that have successfully implemented, sustained, and expanded evidence-based supported employment, known as Individual Placement and Support (IPS), for 4 to 12 years. We interviewed state leaders from 13 states participating in a learning community regarding the composition of their leadership team, participation in the learning community, interagency collaboration, state policy alignment, financing, training, and monitoring of fidelity and outcome. To assess state-level performance in implementing, sustaining, and expanding IPS services, we obtained measures of sustainment, expansion, program fidelity, and employment in the subsequent year and compared them to a priori benchmarks. The majority of states (between 69 % and 77 %) met benchmarks for sustainment, expansion, fidelity, and employment. States varied widely in specific actions to advance IPS, but all had established leadership teams, participated in the national learning community, and built an infrastructure supporting IPS. Leaders in 13 states participating in a learning community have adopted and maintained multiple strategies to sustain and expand evidence-based supported employment at a high level of fidelity with good employment outcomes.

Policy Implications of the Mental Health Treatment Study

The Social Security Administration's Mental Health Treatment Study (MHTS) produced positive mental health, employment, and quality of life outcomes for people on Social Security Disability Insurance (SSDI). The investigators discuss major policy implications. First, because integrated, evidence-based mental health and vocational services produced clinical and societal benefits, the authors recommend further service implementation for this population. Second, because provision of these services did not reduce SSDI rolls, the authors recommend future research on prevention (helping people avoid needing SSDI) rather than rehabilitation (helping beneficiaries leave SSDI). Third, because integrating mental health, vocational, and general medical services was extremely difficult, the authors recommend a multifaceted approach that includes streamlined funding and infrastructure for training and service integration. Fourth, because insurance coverage for people with disabilities during the MHTS (pre-Affordable Care Act) was chaotic, the authors recommend that financing strategies emphasize functional-not just traditional clinical-outcomes.

Hospitalization Risk Before and After Employment Among Adults With Schizophrenia, Bipolar Disorder, or Major Depression

OBJECTIVE

The influence of employment on subsequent psychiatric hospitalization for people with serious mental illness is unclear. This study examined whether unemployed people with serious mental illness were more or less likely to experience psychiatric hospitalization after gaining employment.

METHODS

A secondary analysis was conducted of data from the Mental Health Treatment Study. Two years of prospective employment and psychiatric hospital outcomes were examined for 2,055 adults with schizophrenia, bipolar disorder, or major depression. The analyses examined associations between employment and psychiatric hospitalization via multilevel regression by using time-lagged modeling.

RESULTS

Employment was associated with a lower subsequent three-month risk of psychiatric hospitalization (odds ratio=.65, 95% confidence interval=.50-.84) after the analysis adjusted for baseline characteristics, including previous psychiatric hospitalizations and self-reported physical health.

CONCLUSIONS

Unemployed outpatients with serious mental illness were less likely to experience psychiatric hospitalization after gaining employment.

Development and Testing of the First-Episode Psychosis Services Fidelity Scale

OBJECTIVE

The purpose of this study was to test the reliability and validity of the First-Episode Psychosis Services Fidelity Scale (FEPS-FS) and compare it with similar scales.

METHODS

A fidelity scale was developed from previously identified essential components of first-episode psychosis services. The scale was tested in six programs in two countries and compared with three existing scales.

RESULTS

Program data collection from multiple sources indicated the feasibility and reliability of the FEPS-FS (intraclass correlation coefficient for interrater reliability=.842; 95% confidence interval=.795-.882). Satisfactory programs scored an average of 86% of the maximum total score; the single unsatisfactory program scored 70%. Compared with the other scales, the FEPS-FS has fewer items, but it has the highest proportion of items common to all scales.

CONCLUSIONS

The FEPS-FS is a feasible, compact, reliable, and valid measure of adherence to evidence-based practices for first-episode psychosis services that can be applied to any first-episode psychosis service.