OnTrackNY's learning healthcare system

Using implementation science to operate as a learning health system to improve outcomes in early psychosis

AIM

Early interventions are well understood to improve psychosis outcomes, but their successful implementation remains limited. This article introduces a three-step roadmap for advancing the implementation of evidence-based practices to operate as a learning health system, which can be applied to early interventions for psychosis and is intended for an audience that is relatively new to systematic approaches to implementation.

METHODS

The roadmap is grounded in implementation science, which specializes in methods to promote routine use of evidence-based innovations. The roadmap draws on learning health system principles that call for commitment of leadership, application of evidence, examination of care experiences, and study of health outcomes. Examples are discussed for each roadmap step, emphasizing both data- and stakeholder-related considerations applicable throughout the roadmap.

CONCLUSIONS

Early psychosis care is a promising topic through which to discuss the critical need to move evidence into practice. Despite remarkable advances in early psychosis interventions, population-level impact of those interventions is yet to be realized. By providing an introduction to how implementation science principles can be operationalized in a learning health system and sharing examples from early psychosis care, this article prompts inclusion of a wider audience in essential discourse on the role that implementation science can play for moving evidence into practice for other realms of psychiatric care as well. To this end, the proposed roadmap can serve as a conceptual guiding template and framework through which various psychiatric services can methodically pursue timely implementation of evidence-based interventions for higher quality care and improved outcomes.

Putting measurement-based care into action: A mixed methods study of the benefits of integrating routine client feedback in coordinated specialty care programs for early psychosis

Background

Measurement-based care (MBC) is an effective tool in the delivery of evidence-based practices (EBPs). MBC utilizes feedback loops to share information and drive changes throughout a learning healthcare system. Few studies have demonstrated this practice in team-based care for people with early psychosis. This paper describes the development of a personalized feedback report derived from routine assessments that is shared with clients and clinicians as part of a MBC process.

Methods

We used a quasi pre-post comparison design with mixed methods to evaluate the implementation of a personalized feedback report at 5 early psychosis coordinated specialty care programs (CSC). We compared clients enrolled in CSC who did and did not receive a feedback report over the first 6 months of treatment. The sample included 204 clients: 146 who did not receive the feedback report and were enrolled over 2 years, and 58 who received the feedback report. A subset of 67 clients completed measures at both intake and 6-month follow-up, including 42 who received the report and 25 who did not. We compared the two groups with regard to self-reported symptoms, likelihood of completing treatment, and perception of shared decision making. We conducted qualitative interviews with 5 clients and 5 clinicians to identify the benefits and challenges associated with the personalized feedback report.

Results

People who received a personalized feedback report reported significant improvements in shared decision-making and had greater improvements over time in their intent to attend future treatment sessions. They engaged in more sessions for Supported Employment and Education (SEE), case management, and peer support, and fewer medication visits over the first 6 months of treatment. Both groups showed significant improvement in symptoms and functioning. Results from the qualitative analysis indicated that the experience of receiving the reports was valuable and validating for both patients and clinicians.

Conclusions

A personalized feedback report was integrated into standard of care for early psychosis programs. This process may improve shared decision-making, strengthen the likelihood to stay in treatment, and increase engagement in psychosocial interventions. We posit that this process facilitates strengths-focused discussions, enhances intrinsic motivation, and strengthens the therapeutic alliance.

"Everything Changed, Would You Like Me to Elaborate?": A Qualitative Examination of the Impact of the COVID-19 Pandemic on Community Participation Among Young Adults with Early Psychosis and Their Families

OnTrackNY provides early intervention services to young people with early psychosis throughout New York State. This report describes the impact of the COVID-19 pandemic on community participation of OnTrackNY program participants and their families. Thirteen participants and nine family members participated in five focus groups and three individual semi-structured interviews. Data were analyzed using a summary template and matrix analysis approach. Major themes highlight the negative impacts of the pandemic with reports of decreased socializing or using online means to connect, unemployment, challenges with online learning and a decrease in civic engagement. Positive impacts include more time to deepen connections with family and valued friendships and engage in activities that promote wellness and goal attainment. Implications for coordinated specialty care programs include adapting services to promote mainstream community integration and creating new strategies for community involvement of young people within a new context brought forth by the pandemic.

Incorporating Community Partner Perspectives on eHealth Technology Data Sharing Practices for the California Early Psychosis Intervention Network: Qualitative Focus Group Study With a User-Centered Design Approach

BACKGROUND

Increased use of eHealth technology and user data to drive early identification and intervention algorithms in early psychosis (EP) necessitates the implementation of ethical data use practices to increase user acceptability and trust.

OBJECTIVE

First, the study explored EP community partner perspectives on data sharing best practices, including beliefs, attitudes, and preferences for ethical data sharing and how best to present end-user license agreements (EULAs). Second, we present a test case of adopting a user-centered design approach to develop a EULA protocol consistent with community partner perspectives and priorities.

METHODS

We conducted an exploratory, qualitative, and focus group-based study exploring mental health data sharing and privacy preferences among individuals involved in delivering or receiving EP care within the California Early Psychosis Intervention Network. Key themes were identified through a content analysis of focus group transcripts. Additionally, we conducted workshops using a user-centered design approach to develop a EULA that addresses participant priorities.

RESULTS

In total, 24 participants took part in the study (14 EP providers, 6 clients, and 4 family members). Participants reported being receptive to data sharing despite being acutely aware of widespread third-party sharing across digital domains, the risk of breaches, and motives hidden in the legal language of EULAs. Consequently, they reported feeling a loss of control and a lack of protection over their data. Participants indicated these concerns could be mitigated through user-level control for data sharing with third parties and an understandable, transparent EULA, including multiple presentation modalities, text at no more than an eighth-grade reading level, and a clear definition of key terms. These findings were successfully integrated into the development of a EULA and data opt-in process that resulted in 88.1% (421/478) of clients who reviewed the video agreeing to share data.

CONCLUSIONS

Many of the factors considered pertinent to informing data sharing practices in a mental health setting are consistent among clients, family members, and providers delivering or receiving EP care. These community partners' priorities can be successfully incorporated into developing EULA practices that can lead to high voluntary data sharing rates.

Measures of Shared Decision Making for People With Mental Disorders and Limited Decisional Capacity: A Systematic Review

OBJECTIVE

Shared decision making (SDM) is a health communication model to improve treatment decision making and is underused for people with mental health conditions and limited, impaired, or fluctuating decisional capacity. SDM measures are essential to enhancing the adoption and implementation of SDM practices, yet no tools or research findings exist that explicitly focus on measuring SDM with such patients. The aim of this review was to identify instruments that measure SDM involving individuals with mental health conditions and limited decisional capacity, their family members, and their health and social care providers.

METHODS

A systematic review was performed by searching the PubMed, Embase, Web of Science, and PsycInfo databases. The authors included peer-reviewed, quantitative articles published in English during 2009-2022 that focused on adults (≥18 years old). All authors performed the screening independently.

RESULTS

A total of 7,956 records were identified, six of which met the inclusion criteria for full-text review and five of which were analyzed (one full-text article was not available). No instruments were identified that measured forms of SDM involving patients with mental health conditions and limited, impaired, or fluctuating decisional capacity.

CONCLUSIONS

Measurement instruments to address and assess SDM in health care-related communication processes involving individuals with a mental health condition and limited decisional capacity are needed.

Prediction Tool for Individual Outcome Trajectories Across the Next Year in First-Episode Psychosis in Coordinated Specialty Care

Importance

In coordinated specialty care (CSC) settings for people with a first episode of psychosis, the development of reliable, validated individual-level prediction tools for key outcomes may be informative for shared clinician and client decision-making.

Objective

To develop an individual-level prediction tool using machine-learning methods that predicts a trajectory of education/work status or psychiatric hospitalization outcomes over a client's next year of quarterly follow-up assessments. Additionally, to visualize these predictions in a way that is informative to clinicians and clients.

Design, Setting, and Participants

Individual-level data were collected for all patients enrolled in the OnTrackNY program at enrollment and at quarterly follow-ups using standardized forms. The OnTrackNY program, a network of CSC sites in New York State, provides person-centered, recovery-oriented, and evidence-based psychosocial and pharmaceutical interventions to individuals aged 16 to 30 years with recent-onset (<2 years) nonaffective psychosis. Although data collection is ongoing, data for this study were collected from October 2013 to December 2018, and the time frame for analysis was July 2020 to May 2021. Data were separated into a training/cross-validation set to perform internally validated model development and a separate holdout test set (~20% of the sample) for external validation. Random probability forest models were developed to predict individual-level trajectories of outcomes.

Exposures

Forty-three individual-level demographic and clinical features collected at enrollment in OnTrackNY, 25 of which were time-varying and updated at quarterly follow-up assessments, and 13 site-level demographic and economic census variables.

Main Outcomes and Measures

Individual-level education and/or employment status and psychiatric hospitalization trajectories at quarterly follow-up periods across the first 2 years of CSC.

Results

The total study sample consists of 1298 individuals aged 16 to 30 years and included 341 women (26.3%), 949 men (73.1%), and 8 (<1%) with another gender. Prediction models performed well for 1-year trajectories of education/work across all validation sets, with areas under the receiver operating characteristic curve (AUCs) ranging from 0.68 (95% CI, 0.63-0.74) to 0.88 (95% CI, 0.81-0.96). Predictive accuracy for psychiatric hospitalization 3 months ahead reached AUC above 0.70; moreover, predictions of future psychiatric hospitalizations at 6 months and beyond were consistently poor, with AUCs below 0.60. Given the good externally validated performance for predicting education/work, a prototype interactive visualization tool displaying individual-level education/work trajectories and related features was developed.

Conclusions and Relevance

This study suggests that accurate prediction tools can be developed for outcomes in people with first-episode psychosis, which may help inform shared clinician/client decision-making. Future work should study the effectiveness of its deployment, including proper communication to inform shared clinician/client decision-making in the context of a learning health care system. At present, more work is needed to develop better performing prediction models for future psychiatric hospitalizations before any tool is recommended for this outcome.

Toward Population Health: Using a Learning Behavioral Health System and Measurement-Based Care to Improve Access, Care, Outcomes, and Disparities

Achieving population behavioral health is urgently needed. The mental health system struggles with enormous challenges of providing access to mental health services, improving quality and equitability of care, and ensuring good health outcomes across subpopulations. Little data exists about increasing access within highly constrained resources, staging/sequencing treatment along care pathways, or personalizing treatments. The conceptual model of the learning healthcare system offers a potential paradigm shift for addressing these challenges. In this article we present an overview of how the three constructs of population health, learning health systems, and measurement-based care are inter-related, and we provide an example of how one academic, community-based, safety net health system is approaching integrating these paradigms into its service delivery system. Implementation outcomes will be described in a subsequent publication. We close by discussing how ultimately, to meaningfully improve population behavioral health, a learning healthcare system could expand into a learning health community in order to target critical points of prevention and intervention.

OnTrack Chile for people with early psychosis: a study protocol for a Hybrid Type 1 trial

BACKGROUND

Substantial data from high-income countries support early interventions in the form of evidence-based Coordinated Specialty Care (CSC) for people experiencing First Episode Psychosis (FEP) to ameliorate symptoms and minimize disability. Chile is unique among Latin American countries in providing universal access to FEP services through a national FEP policy that mandates the identification of FEP individuals in primary care and guarantees delivery of community-based FEP treatments within a public health care system. Nonetheless, previous research has documented that FEP services currently provided at mental health clinics do not provide evidence-based approaches. This proposal aims to address this shortfall by first adapting OnTrackNY (OTNY), a CSC program currently being implemented across the USA, into OnTrackChile (OTCH), and then examine its effectiveness and implementation in Chile.

METHODS

The Dynamic Adaptation Process will be used first to inform the adaptation and implementation of OTCH to the Chilean context. Then, a Hybrid Type 1 trial design will test its effectiveness and cost and evaluate its implementation using a cluster-randomized controlled trial (RCT) (N = 300 from 21 outpatient clinics). The OTCH program will be offered in half of these outpatient clinics to individuals ages 15-35. Usual care services will continue to be offered at the other clinics. Given the current COVID-19 pandemic, most research and intervention procedures will be conducted remotely. The study will engage participants over the course of 2 years, with assessments administered at enrollment, 12 months, and 24 months. Primary outcomes include implementation (fidelity, acceptability, and uptake) and service outcomes (person-centeredness, adherence, and retention). Secondary outcomes comprise participant-level outcomes such as symptoms, functioning, and recovery orientation. Over the course of the study, interviews and focus groups with stakeholders will be conducted to better understand the implementation of OTCH.

DISCUSSION

Findings from this study will help determine the feasibility, effectiveness, and cost for delivering CSC services in Chile. Lessons learned about facilitators and barriers related to the implementation of the model could help inform the approach needed for these services to be further expanded throughout Latin America.

TRIAL REGISTRATION

www.

CLINICALTRIALS

gov NCT04247711 . Registered 30 January 2020.

TRIAL STATUS

The OTCH trial is currently recruiting participants. Recruitment started on March 1, 2021, and is expected to be completed by December 1, 2022. This is the first version of this protocol (5/12/2021).

Challenges and Strategies for the Recruitment of Patients With Schizophrenia in a Research Setting

BACKGROUND

With numerous potentially novel targets and pharmacodynamic biomarkers for schizophrenia entering late-stage testing, the next decade will bring an urgent need for well-conducted clinical trials. A critically important step for the successful execution of clinical research trials is timely and appropriate recruitment of participants. Patients with schizophrenia can be especially challenging to recruit because of the disability inherent in psychotic spectrum disorders. Research on how best to recruit for clinical trials is understudied. Clearly defining a model for recruitment procedures would be valuable for researchers and, by extension, the patient populations that may benefit from the insight gained by future clinical research.

METHODS

This article aims to offer suggestions for recruitment based on years of experience at the Columbia Schizophrenia Research Clinic (CSRC), a hub for clinical trials focusing on the etiology and treatment of various psychotic disorders.

RESULTS

The present report provides practical, step-by-step recommendations for implementing the highly effective CSRC recruitment model, including the benefits of 2 recruitment initiatives that were instituted in 2018: hiring a dedicated recruiter and targeted chart reviews at affiliated clinics. Other topics discussed include our umbrella protocol and database, advertising, and tips for collaborating with external sites.

CONCLUSIONS

Despite ongoing complications from coronavirus disease 2019, these strategies have been successful, increasing the rate of both consents and study enrollments by approximately 40% and enabling the CSRC to conduct multiple studies simultaneously.

The Impact of Policy Changes, Dedicated Funding and Implementation Support on Early Intervention Programs for Psychosis

INTRODUCTION

Early intervention services for psychosis (EIS) are associated with improved clinical and economic outcomes. In Quebec, clinicians led the development of EIS from the late 1980s until 2017 when the provincial government announced EIS-specific funding, implementation support and provincial standards. This provides an interesting context to understand the impacts of policy commitments on EIS. Our primary objective was to describe the implementation of EIS three years after this increased political involvement.

METHODS

This cross-sectional descriptive study was conducted in 2020 through a 161-question online survey, modeled after our team's earlier surveys, on the following themes: program characteristics, accessibility, program operations, clinical services, training/supervision, and quality assurance. Descriptive statistics were performed. When relevant, we compared data on programs founded before and after 2017.

RESULTS

Twenty-eight of 33 existing EIS completed the survey. Between 2016 and 2020, the proportion of Quebec's population having access to EIS rose from 46% to 88%; >1,300 yearly admissions were reported by surveyed EIS, surpassing governments' epidemiological estimates. Most programs set accessibility targets; adopted inclusive intake criteria and an open referral policy; engaged in education of referral sources. A wide range of biopsychosocial interventions and assertive outreach were offered by interdisciplinary teams. Administrative/organisational components were less widely implemented, such as clinical/administrative data collection, respecting recommended patient-to-case manager ratios and quality assurance.

CONCLUSION

Increased governmental implementation support including dedicated funding led to widespread implementation of good-quality, accessible EIS. Though some differences were found between programs founded before and after 2017, there was no overall discernible impact of year of implementation. Persisting challenges to collecting data may impede monitoring, data-informed decision-making, and quality improvement. Maintaining fidelity and meeting provincial standards may prove challenging as programs mature and adapt to their catchment area's specificities and as caseloads increase. Governmental incidence estimates may need recalculation considering recent epidemiological data.

Rates of Inpatient and Emergency Room Use Before and After Discharge Among Medicaid Enrollees in OnTrackNY

OBJECTIVE

This study examined hospital and emergency room (ER) use among Medicaid enrollees before and after discharge from OnTrackNY, a coordinated specialty care program for recent-onset psychosis.

METHODS

Medicaid claims data were linked to program data. Inpatient hospitalization, inpatient days, and ER visits were assessed in the 6 months prior to OnTrackNY enrollment and 6 months prior to and after discharge. The sample consisted of 138 participants with continuous Medicaid enrollment during the study.

RESULTS

Inpatient visits significantly declined from the pre-OnTrackNY enrollment period to the predischarge period (β=-1.23, standard error [SE]=0.22, p<0.001), did not significantly change in the first 6 months after discharge (β=0.19, SE=0.26, p=0.48), and remained significantly lower than before OnTrackNY enrollment (β=-1.05, SE=0.20, p<0.001). Similar patterns were observed for inpatient days and ER use.

CONCLUSIONS

ER and hospital use declined during OnTrackNY participation and did not significantly change in the first 6 months after discharge.

Adapting coordinated specialty care in the post-COVID-19 era: study protocol for an integrative mixed-methods study

BACKGROUND

Coordinated Specialty Care (CSC) programs provide evidence-based services for young people with a recent onset of a psychotic disorder. OnTrackNY is a nationally recognized model of CSC treatment in New York state. In 2019, OnTrackNY was awarded a hub within the Early Psychosis Intervention Network (EPINET) to advance its learning health care system (LHS). The OnTrackNY network is comprised of 23 CSC teams across New York state. OnTrack Central, an intermediary organization, provides training and implementation support to OnTrackNY teams. OnTrack Central coordinates a centralized data collection protocol for quality improvement and evaluation of program fidelity and a mechanism to support practice based-research. OnTrackNY sites' breadth coupled with OnTrack Central oversight provides an opportunity to examine the impacts of the COVID-19 crisis in New York State, and supplementary funding was awarded to the OnTrackNY EPINET hub in 2021 for that purpose.

METHODS

This project will examine the implications of modifications to service delivery within the OnTrackNY LHS during and after the COVID-19 crisis. We will use the Framework for Reporting Adaptations and Modification-Enhanced (FRAME) to classify systematically, code, and analyze modifications to CSC services and ascertain their impact. We will utilize integrative mixed methods. Qualitative interviews with multi-level stakeholders (program participants, families, providers, team leaders, agency leaders, trainers (OnTrack Central), and decision-makers at the state and local levels) will be used to understand the process of making decisions, information about modifications to CSC services, and their impact. Analysis of OnTrackNY program data will facilitate examining trends in team staffing and functioning, and participant service utilization and outcomes. Study findings will be summarized in a CSC Model Adaptation Guide, which will identify modifications as fidelity consistent or not, and their impact on service utilization and care outcomes.

DISCUSSION

A CSC Model Adaptation Guide will inform CSC programs, and the state and local mental health authorities to which they are accountable, regarding modifications to CSC services and the impact of these changes on care process, and participant service utilization and outcomes. The guide will also inform the development of tailored technical assistance that CSC programs may need within OnTrackNY, the EPINET network, and CSC programs nationally.

TRIAL REGISTRATION

NCT04021719 , July 16th, 2019.

Adapting Coordinated Specialty Care in the Post-COVID-19 Era: Study Protocol for an Integrative Mixed-methods Study

Background : Coordinated Specialty Care (CSC) programs provide evidence-based services for young people with a recent onset of a psychotic disorder. OnTrackNY is a nationally recognized model of CSC treatment in New York state. In 2019, OnTrackNY was awarded a hub within the Early Psychosis Intervention Network (EPINET) to advance its learning health care system (LHS). The OnTrackNY network is comprised of 23 CSC teams across New York state. OnTrack Central, an intermediary organization, provides training and implementation support to OnTrackNY teams. OnTrack Central coordinates a centralized data collection protocol for quality improvement and evaluation of program fidelity and a mechanism to support practice based-research. OnTrackNY sites’ breadth coupled with OnTrack Central oversight provides an opportunity to examine the impacts of the COVID-19 crisis in New York State. Methods : This project will examine the implications of modifications to service delivery within the OnTrackNY LHS during and after the COVID-19 crisis. We will use the Framework for Reporting Adaptations and Modifications-Enhanced (FRAME) to classify systematically, code, and analyze modifications to CSC services and ascertain their impact. We will utilize integrative mixed methods. Qualitative interviews with multi-level stakeholders (program participants, families, providers, team leaders, agency leaders, trainers (OnTrack Central), and decision-makers at the state and local levels) will be used to understand the process making decisions, information about modifications to CSC services, and their impact. Analysis of OnTrackNY program data will facilitate examining trends in team staffing and functioning, and participant service utilization and outcomes. Study findings will be summarized in a CSC Model Adaptation Guide , which will identify modifications as fidelity consistent or not, and their impact on service utilization and care outcomes. Discussion : A CSC Model Adaptation Guide will inform CSC programs, and the state and local mental health authorities to which they are accountable, regarding modifications to CSC services and the impact of these changes on care process, and participant service utilization and outcomes. The guide will also inform the development of tailored technical assistance that CSC programs may need within OnTrackNY, the EPINET network, and CSC programs nationally. Trial Registration : NCT04021719, July 16 th , 2019.