Standardizing Fidelity to Evidence-Based Practice in the Early Psychosis Intervention Network

OBJECTIVE

This study investigated variations in the measurement of fidelity to coordinated specialty care (CSC) within the Early Psychosis Intervention Network (EPINET), a learning health system that consists of 101 CSC programs within eight hubs. The study investigated the degree to which five fidelity scales could be mapped onto a standard scale.

METHODS

The investigators identified six fidelity scales in use by EPINET participants; examined their item content, scoring, and data sources; and mapped five scales onto the First Episode Psychosis Services Fidelity Scale (FEPS-FS), which is the most widely used scale.

RESULTS

Mapping five fidelity scales onto the FEPS-FS showed that the percentage of FEPS-FS components successfully mapped ranged from 42% to 81%.

CONCLUSIONS

Mapping five scales onto one that uses dichotomous scoring identified the degree of variation in measures and reduced the amount and quality of usable fidelity data. Identifying variations in fidelity measurement is a core function of a learning health system.

Evidence-Based Supported Employment for Individuals With Obsessive-Compulsive Disorder

Individuals with obsessive-compulsive disorder (OCD), a chronic and disabling psychiatric disorder, experience high rates of occupational impairment. OCD symptoms commonly affect individuals' vocational aspirations and result in disability and the need for financial support, problems that are not addressed by current clinical practice guideline recommendations for treating OCD. This Open Forum highlights the need to address occupational impairment caused by OCD and makes the case for formally evaluating whether evidence-based supported employment can help individuals with OCD find and succeed in meaningful work.

Individual placement and support for young adults: One-year outcomes

OBJECTIVE

Many young adults who are unemployed and not in school need support achieving employment goals. Individual Placement and Support (IPS) is an evidence-based employment practice for adults with serious mental illness, but its applicability to young adults with mental health conditions has not been well-researched. The present study prospectively assessed IPS effectiveness in a national sample of young adults enrolled in routine practice settings in the U.S. public mental health system.

METHOD

Nine community agencies in five states participated in a 1-year follow-up study of young adults (aged 16-24) enrolled in IPS services. The study examined three outcomes: retention in services, employment, and education. State fidelity reviewers examined IPS fidelity using a new fidelity scale, the IPS-Y.

RESULTS

In a sample of 111 participants, the mean age was 19.2, 72 (64.9%) had never worked, and 76 (68.5%) had a diagnosis of depressive and/or anxiety disorder. Participants averaged 8 months of enrollment before terminating from IPS services. During follow-up, 51 (45.9%) participants obtained a competitive job (N = 50) or paid internship (N = 1); 14 (12.6%) achieved a new education outcome.

CONCLUSIONS AND IMPLICATIONS FOR PRACTICE

IPS has promising outcomes for helping young adults with mental health conditions achieve positive employment outcomes, but its effectiveness in helping young adults achieve education goals has not been demonstrated. IPS should be offered to young adults with employment goals. Targeted funding for supported education and training for IPS specialists in delivering educational supports may be necessary to ensure optimal education outcomes in IPS. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Individual placement and support programmes for young adults: Where are they and whom do they serve?

AIM

Individual Placement and Support (IPS), an evidence-based supported employment model developed for adults with serious mental illness, has been recently targeted to young adults with mental health conditions, but little is known about its adoption in this age group in the United States.

METHODS

We recruited a volunteer sample of nine IPS programmes in five states serving young adults with mental health conditions aged 16 to 24. IPS team leaders reported programme and participant characteristics and rated barriers to employment and education.

RESULTS

Most IPS programmes were located in community mental health centres, served a small number of young adults, and received most referrals from external sources. The study sample of 111 participants included 53% female, 47% under 21 years old, 60% diagnosed with a depressive disorder; 92% had an employment goal, and 40% had an education goal. IPS specialists reported that managing mental health symptoms was the most common barrier to achieving employment and education goals.

CONCLUSION

Future research should examine how IPS programmes could best provide services to young adults.

Access, retention, and effectiveness of individual placement and support in the US: Are there racial or ethnic differences?

BACKGROUND: Increased federal attention to advancing racial equity and support for underserved communities suggests the need for data on racial and ethnic differences in evidence-based employment services for people with serious mental illness. Individual Placement and Support (IPS) is an evidence-based model of supported employment for this population. OBJECTIVE: The objective was to identify differences based on race and ethnicity in IPS services. METHODS: This narrative review examined the empirical literature on IPS services in the U.S., assessing evidence of differences in access, retention, and outcomes for Black and Hispanic IPS clients, relative to non-Hispanic Whites. RESULTS: We identified 12 studies examining racial and ethnic differences in access (4 studies), retention (3 studies), and effectiveness (6 studies). The findings for access to IPS were mixed, with two studies showing no differences, one finding less access for Blacks, and another finding greater access for Blacks but less access for Hispanics. Three studies found better retention rates for clients enrolled in IPS regardless of race or ethnicity. Compared to clients receiving usual vocational services, all studies found better employment outcomes for IPS clients regardless of race or ethnicity. CONCLUSION: Unlike for most of health care, few racial and ethnic differences have been found for IPS employment services in the U.S. Access to IPS is inadequate for all groups, with conflicting evidence whether Blacks and Hispanics have even less access. Based on the available evidence, Black and Hispanic clients have comparable retention and employment outcomes in IPS as non-Hispanic White clients. State and local mental health leaders responsible for monitoring IPS outcomes should routinely report statistics on race and ethnicity. They should also give active attention to client needs and equity. Research designs should answer multifaceted questions regarding disparities for historically underserved populations.

Modification of the Individual Placement and Support Model of Supported Employment

Individual placement and support (IPS) is a robust evidence-based model of supported employment for people with mental health conditions that has been implemented in high-income countries. The model is now being extended to new populations and settings, often with modifications. Current evidence indicates that minor modifications may increase fit, major alterations of core principles generally reduce effectiveness, and augmentations have mixed success. The authors recommend that those who implement IPS adhere to its core principles whenever possible, use standardized methods to document and assess modifications, and follow basic scientific procedures to develop, measure, and evaluate modifications before recommending policy changes.

A Systematic Review and Meta-analysis of IPS Supported Employment for Young Adults with Mental Health Conditions

Young adults with mental health conditions want to work and advance their education, but many need help attaining these goals. Individual Placement and Support (IPS), originally developed for working-age adults with serious mental illness, is an evidence-based employment model that may benefit young adults. This study is the first systematic review and meta-analysis of randomized controlled trials (RCTs) of IPS for this population. We conducted a systematic review of the effectiveness of IPS for young adults with mental health conditions, supplementing our electronic search of the published literature with secondary analyses of two published RCTs. Using meta-analysis, we evaluated employment rate, job duration, and education rate. Seven studies met the inclusion criteria. Four evaluated IPS for young adults with early psychosis and three evaluated IPS for other young adult subgroups. All found a significantly higher employment rate for IPS than the control group. Overall, 208 (58.3%) of 357 IPS participants and 110 (32.4%) of 340 control participants were competitively employed during follow-up, yielding an overall risk ratio of 1.69 (95% CI 1.43, 1.99), z = 6.24, p < 0.001. Six of the seven studies also reported longer job duration for IPS than the control group, yielding an overall g = 0.34 (95% CI 0.09, 0.58), z = 2.72, p < 0.01. None of four RCTs examining education outcomes found a significant difference favoring IPS, but the overall risk ratio was significant: 1.33 (95% CI 1.06, 1.66), z = 2.51, p < 0.01. Although the empirical literature is limited, IPS appears to be effective in helping young adults with serious mental illness or early psychosis gain and keep competitive jobs. The impact of IPS on education outcomes is unclear. Future research should evaluate the generalizability of these findings to the broad range of young adults with mental health conditions needing help with their employment goals.

Comparing Service Use and Costs of Individual Placement and Support With Usual Vocational Services for Veterans With PTSD

OBJECTIVE

Among veterans with posttraumatic stress disorder (PTSD), supported employment that utilizes the individual placement and support (IPS) model has resulted in consistently better employment and functional outcomes than usual vocational rehabilitation services. This study aimed to compare these two approaches in terms of health services use and associated costs.

METHODS

A secondary analysis of a multisite randomized controlled trial of 541 unemployed veterans with PTSD used archival data from electronic medical records to assess the use and costs of health services of IPS and usual care (i.e., a transitional work [TW] program) over 18 months. Comparisons were also made to an 18-month postintervention period.

RESULTS

The two study groups did not differ in number of inpatient days or in utilization or cost of high-intensity services. Annual per-person costs of health services were approximately 20% higher for IPS than for TW participants (mean difference=$4,910 per person per year, p<0.05) during the intervention period, largely driven by higher utilization and costs for vocational services in the IPS group (p<0.001). These costs declined postintervention to nonsignificant differences. The mean annual per-person vocational service cost was $6,388 for IPS and $2,549 for TW (mean difference=$3,839, p<0.001) during the intervention period.

CONCLUSIONS

In keeping with IPS’s intensive case management approach, veterans receiving IPS used more vocational services and had correspondingly higher costs than veterans receiving TW. The two groups did not differ in use or cost of other types of health services. Future research should examine whether higher short-term costs associated with IPS relative to usual care result in long-term cost savings or higher quality of life for persons with PTSD.

Transition from Military Service: Mental Health and Well-being Among Service Members and Veterans with Service-connected Disabilities

Transitioning from military service is stressful for veterans with service-connected disabilities seeking civilian employment. This descriptive study examined self-assessed mental health, well-being, and substance use of men and women shortly before or after transition from US military service, compared to norms from community and military samples. As part of a prospective study evaluating an innovative employment program, researchers interviewed 229 current and former service members with service-connected disabilities transitioning from U.S. military service. Compared to published norms, respondents reported significantly poorer outcomes on 5 of 6 standardized measures, indicating less life satisfaction, poorer mental health, more symptoms of depression and posttraumatic stress disorder, and greater financial distress. In the previous year, 42% were prescribed opioid medications, over twice the annual opioid prescription rate of 19% in the general US population. Systematic strategies are needed to ensure access for transitioning veterans with serious behavioral health issues to appropriate evidence-based practices.

Evaluation of an Employment Intervention for Veterans Transitioning From the Military: A Randomized Controlled Trial

ABSTRACT

Military personnel face numerous challenges transitioning from military jobs to meaningful civilian employment. The Independence Project compared an innovative employment program (National Career Coach Program) with standard employment services (Local Community Resources) in a randomized controlled trial. Study participants were transitioning veterans with self-reported service-connected disabilities seeking permanent employment. The primary outcomes were paid employment and disability ratings over 1 year. Secondary outcomes included health and well-being. At 1-year follow-up, National Career Coach Program participants were significantly more likely to work, had significantly greater earnings, and reported significantly greater improvements in physical and mental health compared with participants assigned to Local Community Resources. Both groups increased in disability ratings over 12 months, with no difference between groups. Multifaceted supports delivered by the National Career Coach Program increased employment, earnings, mental health, and physical health over 1 year. These significant differences appeared even though control group participants achieved considerable employment success.

Growth of IPS Supported Employment Programs in the United States: An Update

OBJECTIVE

Individual placement and support (IPS), an evidence-based supported employment practice, is a core service in community mental health in the United States. Several factors promote the growth of IPS, including a network of 24 states participating in a learning community devoted to expanding IPS services. This study examined growth of IPS in the United States from 2016 to 2019, comparing growth rates for states within and outside the learning community.

METHODS

This national survey included telephone interviews with 70 representatives from state mental health and vocational rehabilitation agencies in 50 states and the District of Columbia, replicating methods of a 2016 survey. The primary outcomes were the number of IPS programs and clients served. The survey inquired about four indicators of state-level support for IPS implementation and sustainment: collaboration between state agencies, independent fidelity reviews, technical assistance and training, and funding.

RESULTS

In 2019, 41 (80%) of 50 states and the District of Columbia had IPS services, with 857 IPS programs serving an estimated 43,209 clients. Between 2016 and 2019, the number of programs increased from 272 to 486 in 22 learning-community states and two learning-community counties, and from 251 to 371 in 18 states outside the learning community. State-level support for IPS was significantly greater in learning-community states, compared with non-learning-community states.

CONCLUSIONS

IPS services expanded substantially in the United States between 2016 and 2019. Learning-community states had more rapid growth and provided greater implementation support, facilitating implementation, expansion, and sustainment of high-fidelity IPS. Nevertheless, access to IPS remains limited.

Helping People Denied Disability Benefits for a Mental Health Impairment: The Supported Employment Demonstration

Social Security Administration demonstration projects that are intended to help people receiving disability benefits have increased employment but not the number of exits from disability programs. The Supported Employment Demonstration (SED) is a randomized controlled trial (RCT) of services for individuals with mental health problems before they enter disability programs. The SED aims to provide health, employment, and other support services that help them become self-sufficient and avoid entering disability programs. The target population is people who have been denied Social Security disability benefits for a presumed psychiatric impairment. Thirty community-based programs across the United States serve as treatment sites; inclusion in the SED was based on the existence of high-fidelity employment programs that use the individual placement and support model, the ability to implement team-based care, and the willingness to participate in a three-armed RCT. In the SED trial, one-third of 2,960 participants receive services as usual, one-third receive services from a multidisciplinary team that includes integrated supported employment, and one-third receive services from a similar team that also includes a nurse care coordinator for medication management support and medical care. The goals of the study are to help people find employment, attain better health, and delay or avoid disability program entry. This article introduces the SED.

Cost-Effectiveness of Individual Placement and Support Compared to Transitional Work Program for Veterans with Post-traumatic Stress Disorder

Post-traumatic stress disorder (PTSD) leads to significant disability, unemployment, and substantial healthcare costs. The cost-effectiveness of vocational rehabilitation (VR) interventions is important to consider when determining which services to offer. This study assesses the cost-effectiveness and return on investment of Individual Placement and Support (IPS) compared to transitional work (TW) programs. Employment outcomes from a multisite randomized trial comparing IPS to TW in military veterans with PTSD (n = 541) were linked to Veterans Health Administration (VHA) archival medical record databases to examine the comparative cost-effectiveness and return on investment. Effectiveness was defined as hours worked and income earned in competitive jobs. Costs for VR, mental health, and medical care and income earned from competitive sources were annualized and adjusted to 2019 US dollars. The annualized mean cost per person of outpatient (including vocational services) were $3970 higher for IPS compared to TW ($23,245 vs. $19,276, respectively; P = 0.004). When TW income was included in costs, mean grand total costs per person per year were similar between groups ($29,828 IPS vs. $26,772 TW; P = 0.17). The incremental cost-effectiveness analysis showed that while IPS is more costly, it is also more effective. The return on investment (excluding TW income) was 32.9% for IPS ($9762 mean income/$29,691 mean total costs) and 29.6% for TW ($7326 mean income/$24,781 mean total costs). IPS significantly improves employment outcomes for individuals with PTSD with negligible increase in healthcare costs and yields very good return on investment compared to non-IPS VR services.

The effect evaluation of Individual Placement and Support (IPS) for patients with substance use disorders: study protocol for a randomized controlled trial of IPS versus enhanced self-help

Background

Employment is associated with better outcomes of substance use treatment and protects against relapse after treatment completion. Unemployment rates are high for people with substance use disorders (SUD) who undergo treatment, with Norwegian estimates ranging from 81 to 91%. Evidence-based vocational models are lacking for patients in SUD treatment but exist for patients with psychosis in terms of Individual Placement and Support (IPS). The aim of the IPS for substance use disorders (IPS-SUD) trial is to investigate the effect of IPS in a SUD population.

Methods/design

The IPS-SUD trial is a randomized controlled trial (RCT) comparing IPS to an enhanced control intervention. The study is a seven-site, two-arm, pragmatic, parallel-group, superiority RCT. Participants are randomly assigned (1:1) to receive either IPS plus treatment as usual (TAU) or to receive a self-help guide book and 12-h workshop plus 1-h individual vocational guidance plus TAU. Aiming to recruit 200 participants, we will be able to detect a 20% difference in the main outcome of employment with 90% power. We will make assessments at inclusion and at 6- and 12-month follow-ups and obtain outcome data on employment from national mandatory registries. The primary outcome will be at least 1 day of competitive employment during the 18-month follow-up period. Secondary employment outcomes will capture the pattern and extent of employment in terms of total time worked (days/hours), time to first employment, number of different jobs, duration of the longest employment, and sustained employment. Secondary non-employment outcomes will be substance use, mental distress, and quality of life measured by validated instruments at 6, 12, and 18 months follow-up assessments. To be eligible, participants must be between 18 and 65 years, currently unemployed and in treatment for SUD.

Discussion

The IPS-SUD trial will provide evidence for the use of IPS in a SUD population. Findings from the study will have implications for service delivery.

Trial registration

ClinicalTrials.gov NCT04289415. Registered on February 28, 2020

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05673-z.

Psychiatric Crisis Care and the More is Less Paradox

Psychiatric crisis care in the U.S. exemplifies the "more is less paradox" of U.S. health care. We spend more for health care than any other high-income country, yet our outcomes are typically poor compared to these other countries (OECD in OECD health statistics. Retrieved from https://www.oced.org/health/health-data.html , 2020). We do this, in part, by emphasizing medical treatments for problems that are inherently social, rather than addressing social determinants of health. Medical interventions for socio-economic problems are usually expensive and ineffective. For mental health crisis care, adding unfunded, untested, medical interventions to the current mélange of poorly funded, disorganized arrangements will not help. Instead, the U.S. should address social determinants, emphasize research-based interventions, and emphasize prevention-proven strategies that decrease costs and improve outcomes.

COVID-19, Unemployment, and Behavioral Health Conditions: The Need for Supported Employment

The COVID-19 pandemic has caused massive unemployment, exacerbated pre-existing behavioral health (mental health and substance use) disorders for many people, and created new disorders for others. Although policy changes have increased health care and unemployment benefits, most people want jobs and self-sufficiency rather than handouts. A robust evidence base shows that supported employment can enable unemployed people with behavioral health conditions to find competitive, integrated employment and behavioral health supports. Millions of U.S. citizens may need these services as the pandemic recedes and jobs become available. Government attention to supported employment is necessary now more than ever.

Reliability and Feasibility of the First-Episode Psychosis Services Fidelity Scale-Revised for Remote Assessment

OBJECTIVE

The authors sought to evaluate the interrater reliability and feasibility of the First-Episode Psychosis Services Fidelity Scale-Revised (FEPS-FS-R) for remote assessment of first-episode psychosis programs according to the coordinated specialty care model.

METHODS

The authors used the FEPS-FS-R to assess the fidelity of 36 first-episode psychosis program sites in the United States with information from three sources: administrative data, health record review, and phone interviews with staff. Four raters independently conducted fidelity assessments of five program sites by listening to each of the staff interviews and independently rating the two other data sources from each site. To calculate interrater reliability, the authors used intraclass correlation coefficients (ICCs) for each of the five sites and across the total scores for each site.

RESULTS

Total interrater reliability was in the good to excellent range, with a mean ICC of 0.91 (95% confidence interval = 0.72-0.99, p<0.001). Two first-episode psychosis program sites (6%) achieved excellent fidelity, 25 (69%) good fidelity, and nine (25%) fair fidelity. Of the 32 distinct items on the FEPS-FS-R, 23 (72%) were used with good or excellent fidelity. Most sites achieved high fidelity on most items, but five items received ratings indicating low-fidelity use at most sites. The fidelity assessment proved feasible, and sites required on average 10.5 hours for preparing and conducting the fidelity review.

CONCLUSIONS

The FEPS-FS-R has high interrater reliability and can differentiate high-, moderate-, and low-fidelity sites. Most sites had good overall fidelity, but the FEPS-FS-R identified some services that were challenging to implement at many sites.

The Antipsychotic Medication Management Fidelity Scale: Psychometric properties

The paper describes the Antipsychotic Medication Management Fidelity Scale and its psychometric properties, including interrater reliability, frequency distribution, sensitivity to change and feasibility. Fidelity assessors conducted fidelity reviews four times over 18 months at eight sites receiving implementation support for evidence-based antipsychotic medication management. Data analyses shows good to fair interrater reliability, adequate sensitivity to change over time and good feasibility. At 18 months, item ratings varied from poor to full fidelity on most items. Use of the scale can assess fidelity to evidence-based guidelines for antipsychotic medication management and guide efforts to improve practice. Further research should improve and better calibrate some items, and improve the procedures for access to information.Trial registration: ClinicalTrials.gov Identifier: NCT03271242.

The Physical Health Care Fidelity Scale: Psychometric Properties

Mental health programs need an instrument to monitor adherence to evidence-based physical health care for people with serious mental illness. The paper describes the Physical Health Care Fidelity Scale and study interrater reliability, frequency distribution, sensitivity to change and feasibility. Four fidelity assessments were conducted over 18 months at 13 sites randomized to implementation support for evidence-based physical health care. We found good to excellent interrater reliability, adequate sensitivity for change, good feasibility and wide variability in fidelity across sites after 18 months of implementation. Programs were more successful in establishing Policies stating physical health care standards than in implementing these Policies. The Physical Health Care Fidelity Scale measures and guides implementation of evidence-based physical health care reliably.Trial registration: ClinicalTrials.gov Identifier: NCT03271242.

Growth and Sustainment of Individual Placement and Support

Many people with serious mental disorders want to work. Individual placement and support (IPS), an evidence-based program that helps such individuals find and maintain jobs of their choice, has spread steadily across the United States (which has more than 1,000 IPS programs) and to at least 19 other countries during the past 20 years. Four factors explain the program's success. First, IPS is client-centered-IPS specialists follow their clients' preferences regarding issues such as type of employment, style of job search, and type of job support needed. Second, research has undergirded IPS's evolution, helping to refine its effectiveness. Third, support from private foundations and federal agencies has helped to foster implementation, research, and dissemination efforts. Fourth, an international learning community has consolidated cooperation among stakeholders and mental health and vocational rehabilitation agencies and has facilitated continuous quality improvement through data sharing and extensive interactions.

A Tale of Four States: Factors Influencing the Statewide Adoption of IPS

Evidence-based supported employment has become a core community mental health service in much of the U.S. Although a national learning community has facilitated progress in about half of the states, other states have tried to implement evidence-based supported employment on their own. Many studies have examined site-level factors influencing implementation of supported employment, but few have focused on the role of state agency policies and actions. This study examined four states that have not joined the learning community, comparing two that have implemented with success (adopting states) and two that have faced challenges (non-adopting states). This comparative case study approach compared barriers, facilitators, and strategies in two states adopting IPS to two states that did not. The authors examined quantitative data from public records and conducted content analysis of qualitative and quantitative data from key informant interviews. The two non-adopting states lacked model clarity, funding, focus on people with serious mental illness, and collaboration between state mental health and vocational rehabilitation agencies. The two successful states experienced similar barriers but overcame them following lawsuit settlements that required implementation of evidence-based supported employment. Key strategies for successful implementation were funding, fidelity monitoring, technical assistance, and collaboration between state mental health and vocational rehabilitation agencies. With legal settlements serving as the catalyst, states facing challenges to implementing evidence-based supported employment can achieve success using standard implementation strategies to fund and ensure the quality of services.

Evaluating agency-led adaptions to an evidence-based lifestyle intervention for adults with serious mental illness

Background:

Limited empirical evidence exists on the impact of adaptations that occur in implementing evidence-based practices (EBPs) in real-world practice settings. The purpose of this study was to measure and evaluate adaptations to an EBP (InSHAPE) for obesity in persons with serious mental illness in a national implementation in mental health care settings.

Methods:

We conducted telephone interviews with InSHAPE provider teams at 37 (95%) of 39 study sites during 24-month follow-up of a cluster randomized trial of implementation strategies for InSHAPE at behavioral health organizations. Our team rated adaptations as fidelity-consistent or fidelity-inconsistent. Multilevel regression models were used to estimate the relationship between adaptations and implementation and participant outcomes.

Results:

Of 37 sites interviewed, 28 sites (76%) made adaptations to InSHAPE ( M = 2.1, SD = 1.3). Sixteen sites (43%) made fidelity-consistent adaptations, while 22 (60%) made fidelity-inconsistent adaptations. The number of fidelity-inconsistent adaptations was negatively associated with InSHAPE fidelity scores (β = −4.29; p < .05). A greater number of adaptations were associated with significantly higher odds of participant-level cardiovascular risk reduction (odds ratio [ OR] = 1.40; confidence interval [CI] = [1.08, 1.80]; p < .05). With respect to the type of adaptation, we found a significant positive association between the number of fidelity-inconsistent adaptations and cardiovascular risk reduction ( OR = 1.59; CI = [1.01, 2.51]; p < .05). This was largely explained by the fidelity-inconsistent adaptation of holding exercise sessions at the mental health agency versus a fitness facility in the community (a core form of InSHAPE) ( OR = 2.52; 95% CI = [1.11, 5.70]; p < .05).

Conclusions:

This research suggests that adaptations to an evidence-based lifestyle program were common during implementation in real-world mental health practice settings even when fidelity was monitored and reinforced through implementation interventions. Results suggest that adaptations, including those that are fidelity-inconsistent, can be positively associated with improved participant outcomes when they provide a potential practical advantage while maintaining the core function of the intervention.

Plain language abstract:

Treatments that have been proven to work in research studies are not always one-size-fits-all. In real-world clinical settings where people receive mental health care, sometimes there are good reasons to change certain things about a treatment. For example, a particular treatment might not fit well in a specific clinic or cultural context, or it might not meet the needs of specific patient groups. We studied adaptations to an evidence-based practice (InSHAPE) targeting obesity in persons with serious mental illness made by teams implementing the program in routine mental health care settings. We learned that adaptations to InSHAPE were common, and that an adaptation that model experts initially viewed as inconsistent with fidelity to the model turned out to have a positive impact on participant health outcomes. The results of this study may encourage researchers and model experts to work collaboratively with mental health agencies and clinicians implementing evidence-based practices to consider allowing for and guiding adaptations that provide a potential practical advantage while maintaining the core purpose of the intervention.

INdividual Vocational and Educational Support Trial (INVEST) for young people with borderline personality disorder: study protocol for a randomised controlled trial

Background

The clinical onset of borderline personality disorder (BPD) usually occurs in young people (aged 12–25 years) and commonly leads to difficulty achieving and maintaining vocational (education and/or employment) engagement. While current psychosocial interventions lead to improvements in psychopathology, they have little effect upon functioning. Individual Placement and Support (IPS) is a client-driven model that assists individuals with severe mental illness to engage with education and/or employment appropriate to their personal goals, and that provides ongoing support to maintain this engagement. The objective of the INdividual Vocational and Educational Support Trial (INVEST) is to evaluate the effectiveness of adding IPS to an evidence-based early intervention programme for BPD, with the aim of improving vocational outcomes.

Methods/design

INVEST is a single-blind, parallel-groups, randomised controlled trial (RCT). The randomisation is stratified by gender and age and uses random permuted blocks. The interventions are 39 weeks of either IPS, or ‘usual vocational services’ (UVS). Participants will comprise 108 help-seeking young people (aged 15–25 years) with three or more DSM-5 BPD features and a desire to study or work, recruited from the Helping Young People Early (HYPE) early intervention programme for BPD at Orygen, in Melbourne, Australia. All participants will receive the HYPE intervention. After baseline assessment, staff who are blind to the intervention group allocation will conduct assessments at 13, 26, 39 and 52 weeks. At the 52-week primary endpoint, the primary outcome is the number of days in mainstream education/employment since baseline. Secondary outcomes include the cost-effectiveness of the intervention, quality of life, and BPD severity.

Discussion

Current treatments for BPD have little impact on vocational outcomes and enduring functional impairment is prevalent among this patient group. IPS is a targeted functional intervention, which has proven effective in improving vocational outcomes for adults and young people with psychotic disorders. This trial will investigate whether IPS is effective for improving vocational (employment and educational) outcomes among young people with subthreshold or full-syndrome BPD.

Trial registration

Australian New Zealand Clinical Trials Registry, ID: ACTRN12619001220156. 13 September 2019.

Long-Term Follow-Up of a Randomized Trial of Supported Employment for SSDI Beneficiaries With Mental Illness

OBJECTIVE

In this study, the authors assessed the long-term impact of the Mental Health Treatment Study (MHTS), a randomized controlled trial testing the effects of providing 2 years of employment services based on the evidence-based individualized placement and support model to Social Security Disability Insurance (SSDI) recipients with serious mental illness. Treatment recipients also received systematic medication management, supplemental health care supports, and short-term relief from medical continuing disability review by the Social Security Administration (SSA).

METHODS

MHTS site data for 2,160 participants were linked to SSA administrative data from 2011 to 2015, 1 to 5 years after the original study concluded. Univariate and multivariate models were used to assess the MHTS effects on employment, earnings, and disability benefit suspension-termination up to 7 years after services ended.

RESULTS

The analyses showed that the treatment group was more likely than the control group to work, and average earnings among the treatment group increased more over time than earnings among the control group. Disability benefit suspension/termination did not differ between groups.

CONCLUSIONS

Providing the demonstration's package of services and support to SSDI beneficiaries with psychiatric disabilities for up to 2 years may have a long-term impact on employment and earnings. Under the SSDI program as currently structured, however, even after receiving 2 years of evidence-based supported employment and high-quality mental health services, SSDI beneficiaries with psychiatric conditions are unlikely to achieve economic independence within 5 years.

Commentary on special issue on individual placement and support (IPS) international

OBJECTIVE

In this commentary, we identify major themes found in this set of 11 articles, which examine the history of individual placement and support (IPS) adoption within each nation.

METHOD

We summarize major barriers and strategies to achieve to IPS adoption and implementation and briefly discuss IPS research conducted in these countries.

FINDINGS

Faced with many barriers, countries are developing strategies to promote IPS implementation and dissemination. IPS research has consistently found superior employment outcomes for people receiving IPS compared to usual services. Researchers are evaluating the effectiveness of IPS for new populations.

CONCLUSIONS AND IMPLICATIONS FOR PRACTICE

IPS is expanding worldwide, aided by strong leadership, national directives, and the International IPS Learning Community. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Psychometric Properties of the General Organizational Index (GOI): A Measure of Individualization and Quality Improvement to Complement Program Fidelity

To assess the implementation of effective practices, mental health programs need standardized measures. The General Organizational Index (GOI), although widely used for this purpose, has received minimal psychometric research. For this study, we assessed psychometric properties of the GOI scale administered four times over 18 months during the implementation of a new program in 11 sites. The GOI scale demonstrated high levels of interrater reliability (.97), agreement between assessors on item ratings (86% overall), internal consistency (.77–.80 at three time points), sensitivity to change, and feasibility. We conclude that the GOI scale has acceptable psychometric properties, and its use may enhance implementation and research on evidence-based mental health practices.

Trial registration: REK2015/2169. ClinicalTrials.gov Identifier: NCT03271242

Psychometric Properties of a Fidelity Scale for Illness Management and Recovery

This study examined the psychometric properties and feasibility of the Illness Management and Recovery (IMR) Fidelity scale. Despite widespread use of the scale, the psychometric properties have received limited attention. Trained fidelity assessors conducted assessments four times over 18 months at 11 sites implementing IMR. The IMR Fidelity scale showed excellent interrater reliability (.99), interrater item agreement (94%), internal consistency (.91–.95 at three time points), and sensitivity to change. Frequency distributions generally showed that item ratings included the entire range. The IMR Fidelity scale has excellent psychometric properties and should be used to evaluate and guide the implementation of IMR.

Trial registration: ClinicalTrials.gov Identifier: NCT03271242.

Assessing the Fidelity of Evidence-Based Practices: History and Current Status of a Standardized Measurement Methodology

Evidence-based practices are effective only when implemented faithfully. This paper explicates the history, standardization, and methods for developing and validating measures of fidelity. We overviewed the past 20 years of developing fidelity measures, summarized standardization of the development procedures, and described needed psychometric assessments. Fidelity assessment has become the sine qua non of implementation, technical assistance, and research on evidence-based practices. Researchers have established standardized procedures for scale development and psychometric testing. Widescale use of fidelity measurement remains challenging. The implementation of evidence-based practice and the development and validation of fidelity measures are interdependent. International improvements of mental health care will require attention to both.

Measuring implementation: development of the implementation process assessment tool (IPAT)

BACKGROUND

Implementation science comprises a large set of theories suggesting interacting factors at different organisational levels. Development of literature syntheses and frameworks for implementation have contributed to comprehensive descriptions of implementation. However, corresponding instruments for measuring these comprehensive descriptions are currently lacking. The present study aimed to develop an instrument measuring care providers' perceptions of an implementation effort, and to explore the instrument's psychometric properties.

METHODS

Based on existing implementation literature, a questionnaire was designed with items on individual and team factors and on stages of change in an implementation process. The instrument was tested in a Norwegian study on implementation of evidence based practices for psychosis. Item analysis, factor structure, and internal consistency at baseline were examined.

RESULTS

The 27-item Implementation Process Assessment Tool (IPAT) revealed large variation between mean score of the items. The total scale scores were widely dispersed across respondents. Internal consistency for the total scale was high (Cronbach's alpha: .962), and all but one item contributed positively to the construct. The results indicated four underlying constructs: individual stages for behavioural change, individual activities and perceived support, collective readiness and support, and individual perceptions of the intervention.

CONCLUSIONS

The IPAT appears to be a feasible instrument for investigating the implementation process from the perspective of those making the change. It can enable examination of the relative importance of factors thought to be essential for implementation outcomes. It may also provide ongoing feedback for leaders tailoring support for teams to improve implementation. However, further research is needed to detect the instrument's properties later in the implementation process and in different contexts.

TRIAL REGISTRATION

ClinicalTrials.gov code NCT03271242 (retrospective registered September 5, 2017).

A virtual learning collaborative to implement health promotion in routine mental health settings: Protocol for a cluster randomized trial

BACKGROUND

Despite widespread use of learning collaboratives in health care, few randomized trials have evaluated their effectiveness. The primary aim of this cluster randomized implementation trial is to evaluate the effectiveness of a virtual learning collaborative (VLC) in the implementation of a lifestyle intervention for persons with serious mental illness (SMI) in routine mental health settings, compared to standard individual technical assistance.

METHODS

Forty-eight mental health provider organizations from across the United States will be recruited to participate in the trial. The evidence-based practice to be implemented is the InSHAPE health promotion intervention for persons with SMI. Sites will be stratified by size and randomized to receive an 18-month intensive group-based VLC with monthly learning sessions or individual technical assistance with four scheduled conference calls over 18 months. Sites will be enrolled in three blocks of 16 sites each. The primary outcomes are InSHAPE program participation and fidelity, and participant weight loss; secondary outcomes are program operation, program uptake, participant health behaviors of physical activity and nutrition, organizational change, and program sustainment. Implementation outcomes are measured at 3, 6, 12, 18, and 24 months after the program start-up. Participant-level outcomes are measured at fixed intervals every 3 months after each participant enrolls in the study.

DISCUSSION

This study will determine whether VLCs are an effective implementation strategy among resource-limited providers when the new practice necessitates a shift in mission, scope of practice, type of services delivered, and new financing.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT03891368 Registered 25 March 2019, retrospectively registered. https://clinicaltrials.gov/ct2/show/NCT03891368?term=NCT03891368&rank=1.

Individual placement and support for young adults at risk of early work disability (the SEED trial). A randomized controlled trial

Objectives Individual placement and support (IPS) is an effective approach for helping people with severe mental illness gain employment. This study aimed to investigate if IPS can be effectively repurposed to support young adults at risk of early work disability due to various social and health related problems. Methods A randomized controlled trial including 96 young adults (18‒29 years; 68% men) was conducted in Norway. Participants were not in employment, education, or training, received temporary benefits due to social or health-related problems, and were eligible for traditional vocational rehabilitation (TVR). Participants were randomized to IPS (N=50) or TVR (N=46). Self-reported data were collected at baseline and at 6- and 12-months follow-up. The primary outcome was obtaining any paid employment in the competitive labor market during follow-up. Secondary outcomes were physical and mental health, well-being, coping, alcohol consumption, and drug use. Results Significantly more IPS participants obtained competitive employment compared to TVR participants during 12-months follow-up (48% versus 8%; odds ratio 10.39, 95% confidence interval 2.79‒38.68). The IPS group reported significantly better outcomes than the TVR group in subjective health complaints, helplessness, and hopelessness. In post hoc analyses adjusted for baseline and missing data, the IPS group reported significantly better outcomes on these measures in addition to level of disability, optimism about future well-being, and drug use. Conclusions IPS is effective for young adults at risk of early work disability. IPS was superior to TVR in increasing competitive employment and promoted improvements in some non-vocational outcomes. IPS services should be offered to improve employment rates in this vulnerable group.

Expanding Individual Placement and Support to Populations With Conditions and Disorders Other Than Serious Mental Illness

OBJECTIVE

A systematic review of studies of individual placement and support (IPS) for populations other than those with serious mental illness was conducted.

METHODS

The authors searched three electronic databases (PubMed, Web of Science, and Scopus) for studies of IPS and modified IPS. Eligibility criteria for the systematic review included randomized controlled trials with prospective data collection on competitive employment rate and at least 10 study participants from a well-defined population other than people with serious mental illness. Results were compiled for competitive employment rates, IPS fidelity, and other outcomes.

RESULTS

Three clinical groups other than people with serious mental illness have been studied: people with psychiatric disorders other than serious mental illness, people with substance use disorders, and people with musculoskeletal or neurological disorders. Nine controlled trials with a total of 2,902 participants included six trials with people who had psychiatric disorders other than serious mental illness, two with people who had substance use disorders, and one with people who had spinal cord injuries. In eight studies, results for competitive employment rates significantly favored IPS. Meta-analysis yielded an overall weighted odds ratio of 2.23 (95% confidence interval=1.53-3.24, p<.001). Findings for other employment outcomes also favored IPS, but findings on symptom reduction and quality of life were inconsistent. The strongest (and only replicated) findings were for veterans with posttraumatic stress disorder (PTSD). Methodological limitations included small samples, major modifications to IPS fidelity, and short follow-up periods.

CONCLUSIONS

IPS, often with modifications, is a promising employment intervention for several populations in addition to people with serious mental illnesses. The strongest evidence pertains to veterans with PTSD. IPS should be offered to these veterans. Research on other populations, including people with anxiety, depression, substance use disorder, musculoskeletal or neurological conditions, or pain syndromes, needs development, amplification, and replication.

Introducing Individual Placement and Support (IPS) supported employment in Japan

Individual Placement and Support is a standardized model of supported employment, developed initially for people with serious mental disorders and now applied to people with a range of disabilities. More than two dozen randomized controlled trials around the world show that the approach helps a majority of participants to succeed in competitive employment. Individual Placement and Support is spreading rapidly across the USA and in many other high-income countries, with facilitation by an international learning community.

Comparing predictors of employment in Individual Placement and Support: A longitudinal analysis

Individual Placement and Support (IPS) is an evidence-based model of supported employment for people with serious mental illness. We assessed the effects and relative contributions of predictors of employment among IPS recipients using measures of baseline client characteristics, local economic context, and IPS fidelity. A recent work history, less time on the Social Security rolls, greater cognitive functioning, and a lower local unemployment rate were associated with greater probability of employment. The ability of the model to discriminate between outcomes was limited, and substantial improvements in our understanding of IPS employment outcomes will require the study of novel client, environmental, and IPS implementation factors.

Economic, Labor, and Regulatory Moderators of the Effect of Individual Placement and Support Among People With Severe Mental Illness: A Systematic Review and Meta-analysis

As Individual Placement and Support (IPS) has become the international standard for vocational rehabilitation of adults with serious mental illness, researchers must consider the relationship between IPS and local environments. This meta-analysis used mixed-effects meta-regressions to assess the impact of site-level moderators on the likelihood that IPS recipients, compared with recipients of alternative vocational services, achieved competitive employment. Potential moderators included change in gross domestic product (GDP), local unemployment and unionization rates, and indices describing employment protection regulations, level of disability benefits compensation, and efforts to integrate people with disabilities into the workforce. Regulatory moderators represent facilitators and barriers to employment that may reinforce or detract from the effectiveness of IPS. Across 30 sites drawn from 21 randomized controlled trials in 12 countries (33% in the United States), IPS recipients were 2.31 (95% CI 1.99-2.69) times more likely to find competitive employment than recipients of alternative vocational rehabilitation services. The significant competitive-employment rate advantage of IPS over control services increased in the presence of weaker employment protection legislation and integration efforts, and less generous disability benefits. Policy makers should recognize and account for the fact that labor and disability regulations can create an arrangement of incentives that reduces the relative efficacy of supported employment.

Barriers to employment for people with severe mental illness and criminal justice involvement

Background: People with severe mental illness (SMI) and criminal justice involvement (CJI) are able to achieve competitive employment when provided with evidence-based services, but outcomes are modest compared to studies of SMI in general.Aims: To investigate barriers to employment facing people with SMI and CJI receiving employment services.Method: Employment service providers assessed top three barriers to employment for 87 people with SMI and CJI enrolled in a randomized controlled trial comparing a job club program to supported employment. Main barriers were identified and differences between programs were analyzed. Associations between barriers and client background characteristics were investigated.Results: The most common barriers were failure to engage and disengagement from services, followed by substance abuse. Staff from the two employment programs reported similar barriers.Conclusions: Engagement problems were the single most important barrier to employment across programs. Surprisingly, criminal history was rarely mentioned. This may be explained by barriers arising earlier in the process, before achieving employer contact. The results call for recognition and awareness of the importance of motivational issues as well as factors that may inhibit engagement in services, highlighting possible augmentations to evidence-based employment services that may be necessary in the rehabilitation of this patient group.

Barriers and Facilitators to Sustainment of an Evidence-Based Supported Employment Program

Large-scale initiatives to expand evidence-based practices are often poorly implemented and rarely endure. The purpose of this study was to identify the perceived barriers and facilitators to sustainment of an evidence-based supported employment program, Individual Placement and Support (IPS). Within a 2-year prospective study of sustainment among 129 IPS programs in 13 states participating in a national learning community, we interviewed IPS team leaders and coded their responses to semi-structured interviews using a conceptual framework adapted from another large-scale implementation study. Leaders in 122 agencies (95%) that sustained their IPS programs identified funding, prioritization, and workforce characteristics as both key facilitators and barriers. Additional key factors were lack of local community supports as a barrier and leadership and structured workflow as facilitators. Within the IPS learning community, team leaders attributed the sustainment of their program to funding, prioritization, workforce, agency leadership, and structured workflow. The actions of the learning community's leadership, state governments, and local programs together may have contributed to the high sustainment rate.

Development of a measure of model fidelity for mental health Crisis Resolution Teams

BACKGROUND

Crisis Resolution Teams (CRTs) provide short-term intensive home treatment to people experiencing mental health crisis. Trial evidence suggests CRTs can be effective at reducing hospital admissions and increasing satisfaction with acute care. When scaled up to national level however, CRT implementation and outcomes have been variable. We aimed to develop and test a fidelity scale to assess adherence to a model of best practice for CRTs, based on best available evidence.

METHODS

A concept mapping process was used to develop a CRT fidelity scale. Participants (n = 68) from a range of stakeholder groups prioritised and grouped statements (n = 72) about important components of the CRT model, generated from a literature review, national survey and qualitative interviews. These data were analysed using Ariadne software and the resultant cluster solution informed item selection for a CRT fidelity scale. Operational criteria and scoring anchor points were developed for each item. The CORE CRT fidelity scale was then piloted in 75 CRTs in the UK to assess the range of scores achieved and feasibility for use in a 1-day fidelity review process. Trained reviewers (n = 16) rated CRT service fidelity in a vignette exercise to test the scale's inter-rater reliability.

RESULTS

There were high levels of agreement within and between stakeholder groups regarding the most important components of the CRT model. A 39-item measure of CRT model fidelity was developed. Piloting indicated that the scale was feasible for use to assess CRT model fidelity and had good face validity. The wide range of item scores and total scores across CRT services in the pilot demonstrate the measure can distinguish lower and higher fidelity services. Moderately good inter-rater reliability was found, with an estimated correlation between individual ratings of 0.65 (95% CI: 0.54 to 0.76).

CONCLUSIONS

The CORE CRT Fidelity Scale has been developed through a rigorous and systematic process. Promising initial testing indicates its value in assessing adherence to a model of CRT best practice and to support service improvement monitoring and planning. Further research is required to establish its psychometric properties and international applicability.

Sustaining and Expanding Evidence-Based Supported Employment: The Role of State Leaders Participating in a Learning Community

State leaders often promote implementation of evidence-based practices but have difficulty sustaining and expanding them over time. This paper examines the activities of leaders in 13 states that have successfully implemented, sustained, and expanded evidence-based supported employment, known as Individual Placement and Support (IPS), for 4 to 12 years. We interviewed state leaders from 13 states participating in a learning community regarding the composition of their leadership team, participation in the learning community, interagency collaboration, state policy alignment, financing, training, and monitoring of fidelity and outcome. To assess state-level performance in implementing, sustaining, and expanding IPS services, we obtained measures of sustainment, expansion, program fidelity, and employment in the subsequent year and compared them to a priori benchmarks. The majority of states (between 69 % and 77 %) met benchmarks for sustainment, expansion, fidelity, and employment. States varied widely in specific actions to advance IPS, but all had established leadership teams, participated in the national learning community, and built an infrastructure supporting IPS. Leaders in 13 states participating in a learning community have adopted and maintained multiple strategies to sustain and expand evidence-based supported employment at a high level of fidelity with good employment outcomes.

Policy Implications of the Mental Health Treatment Study

The Social Security Administration's Mental Health Treatment Study (MHTS) produced positive mental health, employment, and quality of life outcomes for people on Social Security Disability Insurance (SSDI). The investigators discuss major policy implications. First, because integrated, evidence-based mental health and vocational services produced clinical and societal benefits, the authors recommend further service implementation for this population. Second, because provision of these services did not reduce SSDI rolls, the authors recommend future research on prevention (helping people avoid needing SSDI) rather than rehabilitation (helping beneficiaries leave SSDI). Third, because integrating mental health, vocational, and general medical services was extremely difficult, the authors recommend a multifaceted approach that includes streamlined funding and infrastructure for training and service integration. Fourth, because insurance coverage for people with disabilities during the MHTS (pre-Affordable Care Act) was chaotic, the authors recommend that financing strategies emphasize functional-not just traditional clinical-outcomes.

Hospitalization Risk Before and After Employment Among Adults With Schizophrenia, Bipolar Disorder, or Major Depression

OBJECTIVE

The influence of employment on subsequent psychiatric hospitalization for people with serious mental illness is unclear. This study examined whether unemployed people with serious mental illness were more or less likely to experience psychiatric hospitalization after gaining employment.

METHODS

A secondary analysis was conducted of data from the Mental Health Treatment Study. Two years of prospective employment and psychiatric hospital outcomes were examined for 2,055 adults with schizophrenia, bipolar disorder, or major depression. The analyses examined associations between employment and psychiatric hospitalization via multilevel regression by using time-lagged modeling.

RESULTS

Employment was associated with a lower subsequent three-month risk of psychiatric hospitalization (odds ratio=.65, 95% confidence interval=.50-.84) after the analysis adjusted for baseline characteristics, including previous psychiatric hospitalizations and self-reported physical health.

CONCLUSIONS

Unemployed outpatients with serious mental illness were less likely to experience psychiatric hospitalization after gaining employment.

Development and Testing of the First-Episode Psychosis Services Fidelity Scale

OBJECTIVE

The purpose of this study was to test the reliability and validity of the First-Episode Psychosis Services Fidelity Scale (FEPS-FS) and compare it with similar scales.

METHODS

A fidelity scale was developed from previously identified essential components of first-episode psychosis services. The scale was tested in six programs in two countries and compared with three existing scales.

RESULTS

Program data collection from multiple sources indicated the feasibility and reliability of the FEPS-FS (intraclass correlation coefficient for interrater reliability=.842; 95% confidence interval=.795-.882). Satisfactory programs scored an average of 86% of the maximum total score; the single unsatisfactory program scored 70%. Compared with the other scales, the FEPS-FS has fewer items, but it has the highest proportion of items common to all scales.

CONCLUSIONS

The FEPS-FS is a feasible, compact, reliable, and valid measure of adherence to evidence-based practices for first-episode psychosis services that can be applied to any first-episode psychosis service.