You never know what you are going to get: Large-scale assessment of therapists’ supportive counseling skill use

Supportive counseling skills like empathy and active listening are critical ingredients of all psychotherapies, but most research relies on client or therapist reports of the treatment process. This study utilized machine-learning models trained to evaluate counseling skills to evaluate supportive skill use in 3,917 session recordings. We analyzed overall skill use and variation in practice patterns using a series of mixed effects models. On average, therapists scored moderately high on observer-rated empathy (i.e., 3.8 out of 5), 3.3% of the therapists' utterances in a session were open questions, and 12.9% of their utterances were reflections. However, there were substantial differences in skill use across therapists as well as across clients within-therapist caseloads. These findings highlight the substantial variability in the process of counseling that clients may experience when they access psychotherapy. We discuss findings in the context of both the need for therapists to be responsive and flexible with their clients, but also potential costs related to the lack of a more uniform experience of care. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

High rates of oral anticoagulation in atrial fibrillation patients observed in a large multi-specialty health system in the Northeast

BACKGROUND

Anticoagulation is a cornerstone in atrial fibrillation (AF) management for stroke prevention. Studies showed that oral anticoagulants (OAC), previously limited to warfarin, were underused. Recently, non-vitamin K oral anticoagulants (NOACs) have seen widespread adoption, but it has not been well studied whether there has been a subsequent increase in OAC usage in AF patients.

METHODS

We quantified OAC rates in AF patients in a large multispecialty health system in the Northeast United States. A total of 351,795 patients seen in the network over the preceding 18 months were reviewed.

RESULTS

Of these patients, 8727 (2.5%) carried a diagnosis of AF, and, of the 6933 patients with a CHA₂DS₂-VASc score of 2 or higher, 5576 (80.4%) had an OAC listed as an active medication or had received a left atrial appendage occlusion device. Of the 6605 patients treated with an OAC, 5308 (80.4%) were treated with a NOAC and 1295 (19.6%) were prescribed warfarin. A higher percentage of patients with CHA₂DS₂-VASc ≥ 2 who had seen a cardiologist were treated with an OAC vs. those who had not seen a cardiologist in the prior 18 months (83.95% vs. 67.43%, p < 0.01).

CONCLUSIONS

We show dramatically increased OAC usage among patients with AF and that NOACs comprise the large majority of OACs compared with previous studies. This suggests an association between widespread adoption of NOACs and increased oral anticoagulation rates. Future directions include assessing barriers to oral anticoagulation and developing interventions to reduce disparity in OAC use between clinics.

How the VA is training the Next‐Generation workforce for learning health systems

Objectives

Methods

Results

Conclusions

Developing and Implementing Predictive Models in a Learning Healthcare System: Traditional and Artificial Intelligence Approaches in the Veterans Health Administration

Predicting clinical risk is an important part of healthcare and can inform decisions about treatments, preventive interventions, and provision of extra services. The field of predictive models has been revolutionized over the past two decades by electronic health record data; the ability to link such data with other demographic, socioeconomic, and geographic information; the availability of high-capacity computing; and new machine learning and artificial intelligence methods for extracting insights from complex datasets. These advances have produced a new generation of computerized predictive models, but debate continues about their development, reporting, validation, evaluation, and implementation. In this review we reflect on more than 10 years of experience at the Veterans Health Administration, the largest integrated healthcare system in the United States, in developing, testing, and implementing such models at scale. We report lessons from the implementation of national risk prediction models and suggest an agenda for research. Expected final online publication date for the Annual Review of Biomedical Data Science, Volume 5 is August 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

What Is the Return on Investment of Caring for Complex High-need, High-cost Patients?

Randomized controlled trials to improve care for complex, high-need, high-cost patients have not consistently demonstrated a relative decrease in acute care utilization or cost savings. However, the Veterans Health Administration (VHA) has been able to glean lessons from these trials and generate realistic expectations for success. Lessons include the following: (1) combining population management tools (e.g., risk scores) and clinician judgment is more effective than either alone to identify the patients best suited for intensive management; (2) treatment adherence and engagement may contribute more to preventable emergency department visits and hospitalizations than care coordination; and (3) efforts should focus on assessing for and treating those risk factors that are most amenable to intervention. Because it is unlikely that cost savings can fund add-on intensive management programs, the VHA Office of Primary Care plans to incorporate those intensive management practices that are feasible into existing patient-centered medical homes as a high reliability organization.

Outcomes of a randomized quality improvement trial for high-risk Veterans in year two

OBJECTIVE

The Veterans Health Administration (VHA) conducted a randomized quality improvement evaluation to determine whether augmenting patient-centered medical homes with Primary care Intensive Management (PIM) decreased utilization of acute care and health care costs among patients at high risk for hospitalization. PIM was cost-neutral in the first year; we analyzed changes in utilization and costs in the second year.

DATA SOURCES

VHA administrative data for five demonstration sites from August 2013 to March 2019.

DATA SOURCES

Administrative data extracted from VHA's Corporate Data Warehouse.

STUDY DESIGN

Veterans with a risk of 90-day hospitalization in the top 10th percentile and recent hospitalization or emergency department (ED) visit were randomly assigned to usual primary care vs primary care augmented by PIM. PIM included interdisciplinary teams, comprehensive patient assessment, intensive case management, and care coordination services. We compared the change in mean VHA inpatient and outpatient utilization and costs (including PIM expenses) per patient for the 12-month period before randomization and 13-24 months after randomization for PIM vs usual care using difference-in-differences.

PRINCIPAL FINDINGS

Both PIM patients (n = 1902) and usual care patients (n = 1882) had a mean of 5.6 chronic conditions. PIM patients had a greater number of primary care visits compared to those in usual care (mean 4.6 visits/patient/year vs 3.7 visits/patient/year, p < 0.05), but ED visits (p = 0.45) and hospitalizations (p = 0.95) were not significantly different. We found a small relative increase in outpatient costs among PIM patients compared to those in usual care (mean difference + $928/patient/year, p = 0.053), but no significant differences in mean inpatient costs (+$245/patient/year, p = 0.97). Total mean health care costs were similar between the two groups during the second year (mean difference + $1479/patient/year, p = 0.73).

CONCLUSIONS

Approaches that target patients solely based on the high risk of hospitalization are unlikely to reduce acute care use or total costs in VHA, which already offers patient-centered medical homes.

Soil fertility interactions with Sinorhizobium-legume symbiosis in a simulated Martian regolith; effects on nitrogen content and plant health

Due to increasing population growth and declining arable land on Earth, astroagriculture will be vital to terraform Martian regolith for settlement. Nodulating plants and their N-fixing symbionts may play a role in increasing Martian soil fertility. On Earth, clover (Melilotus officinalis) forms a symbiotic relationship with the N-fixing bacteria Sinorhizobium meliloti; clover has been previously grown in simulated regolith yet without bacterial inoculation. In this study, we inoculated clover with S. meliloti grown in potting soil and regolith to test the hypothesis that plants grown in regolith can form the same symbiotic associations as in soils and to determine if greater plant biomass occurs in the presence of S. meliloti regardless of growth media. We also examined soil NH₄ concentrations to evaluate soil augmentation properties of nodulating plants and symbionts. Greater biomass occurred in inoculated compared to uninoculated groups; the inoculated average biomass in potting mix and regolith (2.23 and 0.29 g, respectively) was greater than the uninoculated group (0.11 and 0.01 g, respectively). However, no significant differences existed in NH₄ composition between potting mix and regolith simulant. Linear regression analysis results showed that: i) symbiotic plant-bacteria relationships differed between regolith and potting mix, with plant biomass positively correlated to regolith-bacteria interactions; and, ii) NH₄ production was limited to plant uptake yet the relationships in regolith and potting mix were similar. It is promising that plant-legume symbiosis is a possibility for Martian soil colonization.

Assessing and communicating heterogeneity of treatment effects for patient subpopulations: Keynote and panel discussion on communicating heterogeneous treatment effects across populations

How do we communicate nuanced regulatory information to different audiences, recognizing that the consumer audience is very different from the physician audience? In particular, how do we communicate the heterogeneity of treatment effects - the potential differences in treatment effects based on sex, race, and age? That is a fundamental question at the heart of this panel discussion. Each panelist addressed a specific "challenge question" during their 5-minute presentation, and the list of questions is provided. The presentations were followed by a question and answer session with members of the audience and the panelists.

Learning health systems: Driving real-world impact in mental health and substance use disorder research

The Veterans Health Administration (VHA), under the U.S. Department of Veterans Affairs (VA), is one of the largest single providers of health care in the U.S. VA supports an embedded research program that addresses VA clinical priorities in close partnership with operations leaders, which is a hallmark of a Learning Health System (LHS). Using the LHS framework, we describe current VA research initiatives in mental health and substance use disorders that rigorously evaluate national programs and policies designed to reduce the risk of suicide and opioid use disorder (data to knowledge); test implementation strategies to improve the spread of effective programs for Veterans at risk of suicide or opioid use disorder (knowledge to performance); and identify novel research directions in suicide prevention and opioid/pain treatments emanating from implementation and quality improvement research (performance to data). Lessons learned are encapsulated into best practices for building and sustaining an LHS within health systems, including the need for early engagement with clinical leaders; pragmatic research questions that focus on continuous improvement; multi‐level, ongoing input from regional and local stakeholders, and business case analyses to inform ongoing investment in sustainable infrastructure to maintain the research‐health system partnership. Essential ingredients for supporting VA as an LHS include data and information sharing capacity, protected time for researchers and leaders, and governance structures to enhance health system ownership of research findings. For researchers, incentives to work with health systems operations (e.g., retainer funding) are vital for LHS research to be recognized and valued by academic promotion committees.

Early initiation of prophylactic anticoagulation for prevention of coronavirus disease 2019 mortality in patients admitted to hospital in the United States: cohort study

OBJECTIVE

To evaluate whether early initiation of prophylactic anticoagulation compared with no anticoagulation was associated with decreased risk of death among patients admitted to hospital with coronavirus disease 2019 (covid-19) in the United States.

DESIGN

Observational cohort study.

SETTING

Nationwide cohort of patients receiving care in the Department of Veterans Affairs, a large integrated national healthcare system.

PARTICIPANTS

All 4297 patients admitted to hospital from 1 March to 31 July 2020 with laboratory confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and without a history of anticoagulation.

MAIN OUTCOME MEASURES

The main outcome was 30 day mortality. Secondary outcomes were inpatient mortality, initiating therapeutic anticoagulation (a proxy for clinical deterioration, including thromboembolic events), and bleeding that required transfusion.

RESULTS

Of 4297 patients admitted to hospital with covid-19, 3627 (84.4%) received prophylactic anticoagulation within 24 hours of admission. More than 99% (n=3600) of treated patients received subcutaneous heparin or enoxaparin. 622 deaths occurred within 30 days of hospital admission, 513 among those who received prophylactic anticoagulation. Most deaths (510/622, 82%) occurred during hospital stay. Using inverse probability of treatment weighted analyses, the cumulative incidence of mortality at 30 days was 14.3% (95% confidence interval 13.1% to 15.5%) among those who received prophylactic anticoagulation and 18.7% (15.1% to 22.9%) among those who did not. Compared with patients who did not receive prophylactic anticoagulation, those who did had a 27% decreased risk for 30 day mortality (hazard ratio 0.73, 95% confidence interval 0.66 to 0.81). Similar associations were found for inpatient mortality and initiation of therapeutic anticoagulation. Receipt of prophylactic anticoagulation was not associated with increased risk of bleeding that required transfusion (hazard ratio 0.87, 0.71 to 1.05). Quantitative bias analysis showed that results were robust to unmeasured confounding (e-value lower 95% confidence interval 1.77 for 30 day mortality). Results persisted in several sensitivity analyses.

CONCLUSIONS

Early initiation of prophylactic anticoagulation compared with no anticoagulation among patients admitted to hospital with covid-19 was associated with a decreased risk of 30 day mortality and no increased risk of serious bleeding events. These findings provide strong real world evidence to support guidelines recommending the use of prophylactic anticoagulation as initial treatment for patients with covid-19 on hospital admission.

Mobile RDoC: Using Smartphones to Understand the Relationship Between Auditory Verbal Hallucinations and Need for Care

Objective

Auditory verbal hallucinations (AVH) are common in multiple clinical populations but also occur in individuals who are otherwise considered healthy. Adopting the National Institute of Mental Health's Research Domain Criteria (RDoC) framework, the aim of the current study was to integrate a variety of measures to evaluate whether AVH experience varies across clinical and nonclinical individuals.

Methods

A total of 384 people with AVH from 41 US states participated in the study; 295 participants (77%) who received inpatient, outpatient, or combination treatments for AVH and 89 participants (23%) who never received care. Participants used a multi-modal smartphone data collection system to report on their AVH experiences and co-occurring psychological states multiple times daily, over 30 days. In parallel, smartphone sensors recorded their physical activity, geolocation, and calling and texting behavior continuously.

Results

The clinical sample experienced AVH more frequently than the nonclinical group and rated their AVH as significantly louder and more powerful. They experienced more co-occurring negative affect and were more socially withdrawn, spending significantly more time at home and significantly less time near other people. Participants with a history of inpatient care also rated their AVH as infused with significantly more negative content. The groups did not differ in their physical activity or use of their smartphones for digital communication.

Conclusion

Smartphone-assisted remote data collection revealed real-time/real-place phenomenological, affective, and behavioral differences between clinical and nonclinical samples of people who experience AVH. The study provided strong support for the application of RDoC-informed approaches in psychosis research.

Accelerating implementation of research in Learning Health Systems: Lessons learned from VA Health Services Research and NCATS Clinical Science Translation Award programs

Translation of research to practice is challenging. In addition to the scientific challenges, there are additional hurdles in navigating the rapidly changing US health care system. There is a need for innovative health interventions that can be adopted in "real-world" settings. Barriers to translation involve misaligned timing of research funding and health system decision-making, lack of research questions aligned with health system and community priorities, and limited incentives in academia for health system and community-based research. We describe new programs from the US Department of Veterans Affairs Health Services Research and Development (HSR&D) and the National Center for Advancing Translational Sciences (NCATS) Clinical and Translational Science Award (CTSA) Programs that are building capacity for Learning Health System research. These programs help to incentivize adopting and adapting Learning Health System principles to ensure that, primarily in implementation science within academic/veterans affairs health systems, there is alignment of the research with the health system and community needs. Both HSR&D and NCATS CTSA Program encourage researchers to develop problem-focused research innovations in partnership with health systems and communities to ultimately facilitate design treatments that are feasible in "real-world" practice.

REPRINT OF: Current Methods of the U.S. Preventive Services Task Force: A Review of the Process

Editor's Note: This article is a reprint of a previously published article. For citation purposes, please use the original publication details: Harris RP, Helfand M, Woolf SH, et al. Current methods of the U.S. Preventive Services Task Force: a review of the process. Am J Prev Med. 2001;20(3S):21-35. The U.S. Preventive Services Task Force (USPSTF/Task Force) represents one of several efforts to take a more evidence-based approach to the development of clinical practice guidelines. As methods have matured for assembling and reviewing evidence and for translating evidence into guidelines, so too have the methods of the USPSTF. This paper summarizes the current methods of the third USPSTF, supported by the Agency for Healthcare Research and Quality (AHRQ) and two of the AHRQ Evidence-based Practice Centers (EPCs). The Task Force limits the topics it reviews to those conditions that cause a large burden of suffering to society and that also have available a potentially effective preventive service. It focuses its reviews on the questions and evidence most critical to making a recommendation. It uses analytic frameworks to specify the linkages and key questions connecting the preventive service with health outcomes. These linkages, together with explicit inclusion criteria, guide the literature searches for admissible evidence. Once assembled, admissible evidence is reviewed at three strata: (1) the individual study, (2) the body of evidence concerning a single linkage in the analytic framework, and (3) the body of evidence concerning the entire preventive service. For each stratum, the Task Force uses explicit criteria as general guidelines to assign one of three grades of evidence: good, fair, or poor. Good or fair quality evidence for the entire preventive service must include studies of sufficient design and quality to provide an unbroken chain of evidence-supported linkages, generalizable to the general primary care population, that connect the preventive service with health outcomes. Poor evidence contains a formidable break in the evidence chain such that the connection between the preventive service and health outcomes is uncertain. For services supported by overall good or fair evidence, the Task Force uses outcomes tables to help categorize the magnitude of benefits, harms, and net benefit from implementation of the preventive service into one of four categories: substantial, moderate, small, or zero/negative. The Task Force uses its assessment of the evidence and magnitude of net benefit to make a recommendation, coded as a letter: from A (strongly recommended) to D (recommend against). It gives an I recommendation in situations in which the evidence is insufficient to determine net benefit. The third Task Force and the EPCs will continue to examine a variety of methodologic issues and document work group progress in future communications.

Coordinating Care Across VA Providers and Settings: Policy and Research Recommendations from VA's State of the Art Conference

Delivering well-coordinated care is essential for optimizing clinical outcomes, enhancing patient care experiences, minimizing costs, and increasing provider satisfaction. The Veterans Health Administration (VA) has built a strong foundation for internally coordinating care. However, VA faces mounting internal care coordination challenges due to growth in the number of Veterans using VA care, high complexity in Veterans' care needs, the breadth and depth of VA services, and increasing use of virtual care. VA's Health Services Research and Development service with the Office of Research and Development held a conference assessing the state-of-the-art (SOTA) on care coordination. One workgroup within the SOTA focused on coordination between VA providers for high-need Veterans, including (1) Veterans with multiple chronic conditions; (2) Veterans with high-intensity, focused, specialty care needs; (3) Veterans experiencing care transitions; (4) Veterans with severe mental illness; (5) and Veterans with homelessness and/or substance use disorders. We report on this workgroup's recommendations for policy and organizational initiatives and identify questions for further research. Recommendations from a separate workgroup on coordinating VA and non-VA care are contained in a companion paper. Leaders from research, clinical services, and VA policy will need to partner closely as they develop, implement, assess, and spread effective practices if VA is to fully realize its potential for delivering highly coordinated care to every Veteran.

HSR Commentary: Linking VA and Non-VA Data to Address Important US Veteran Health Services Research Issues

OBJECTIVE

This commentary summarizes the methods and topics addressed in the special issue of HSR focused on linkage of United States Department of Veterans Affairs (VA) and non-VA datasets. The issue illustrates that researchers are increasingly linking diverse datasets as a valuable method for obtaining outcomes, treatments, and covariates to evaluate and examine health care delivery that includes non-VA services. The issue serves as a reference to VA and non-VA investigators alike who employ data linkage methods to address high-impact clinical and health policy evaluations that span different care systems and different datasets.

Noncommercial US Funders' Policies on Trial Registration, Access to Summary Results, and Individual Patient Data Availability

IMPORTANCE

Incomplete information about existing research is an underlying cause of research waste. National and international initiatives and requirements have been launched to address this issue.

OBJECTIVES

To characterize current clinical trial transparency policies among the largest noncommercial US funders and examine whether the policies are concordant with international funders.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective review of public information used methods developed for documenting funder policies internationally; 2 researchers searched each funder's website and Google between May and November 2018 to locate trial transparency policies for 10 top US funders. Key informants at each funding organization were contacted by email and given 3 or more weeks to review and confirm or correct the findings. Nonresponders were contacted 2 or more additional times. Descriptive statistics were calculated to summarize the findings. The study was conducted using publicly available policy information with findings confirmed by funder representatives where possible. Participants included top 10 noncommercial US health research funders with the highest reported investment in health research (2013 dollars) who fund clinical trials. Data analysis was conducted from November 6, 2018, to November 23, 2018.

EXPOSURES

Availability of policies addressing each of the 3 key trial transparency domains as specified by the World Health Organization in 2017.

MAIN OUTCOMES AND MEASURES

Independent assessment by 2 investigators of availability (yes or no) of a policy addressing registration for trials, sharing of summary results, and individual participant data sharing activities; requirements (yes, no, or supportive statement) of these policies in terms of completeness, timeliness, public access, and provision of additional technical or financial support to meet data sharing requirements; description (yes or no) of internal monitoring for policy adherence.

RESULTS

All 10 funders acknowledged the outreach. One funder who indicated that less than 1% of their research funding goes to clinical trials was removed. Six (67%) of the remaining 9 top US funders have a publicly available written policy for all 3 major trial transparency domains. The most comprehensive trial transparency practice among US funders addresses summary results sharing as follows: 8 of 9 US funders (89%) have a policy, 5 of 9 US funders (56%) require reporting of summary results within 1 year, and 6 of 9 US funders (67%) monitor compliance with their summary results sharing policy. For clinical trial registration, 7 of 9 US funders (78%) have a policy and 5 of 9 US funders (56%) require registration and monitor trial registration to measure adherence to the policy.

CONCLUSIONS AND RELEVANCE

In this study, overall the proportion of US funders with policies and practices to support trial transparency in this sample was similar or compared favorably with the larger international sample of noncommercial funders recently reported.

Overcoming Challenges to Evidence-Based Policy Development in a Large, Integrated Delivery System

OBJECTIVE

To describe a new Veterans Health Administration (VHA) program to foster the learning health system paradigm by rigorously evaluating health care initiatives and to report key lessons learned in designing those evaluations.

PRINCIPAL FINDINGS

The VHA's Quality Enhancement Research Initiative and its Health Services Research and Development Service are cooperating on several large, randomized program evaluations aimed at improving the care veterans receive and the efficiency with which it is delivered. The evaluations we describe involve collaborative design, outcomes assessment, and implementation science through partnerships between VHA operations and researchers. We review key factors to assess before committing to an evaluation. In addition to traditional design issues (such as ensuring adequate power and availability of data), these include others that are easily overlooked: the stability of intervention financing, means of controlling and commitment to adhering to randomized roll-out, degree of buy-in from key implementation staff, and feasibility of managing multiple veto points for interventions that span several programs, among others.

CONCLUSIONS

Successful program implementation and rigorous evaluation require resources, specialized expertise, and careful planning. If the learning health system model is to be sustained, organizations will need dedicated programs to prioritize resources and continuously adapt evaluation designs.

Computational modeling of conversational humor in psychotherapy

Humor is an important social construct that serves several roles in human communication. Though subjective, it is culturally ubiquitous and is often used to diffuse tension, specially in intense conversations such as those in psychotherapy sessions. Automatic recognition of humor has been of considerable interest in the natural language processing community thanks to its relevance in conversational agents. In this work, we present a model for humor recognition in Motivational Interviewing based psychotherapy sessions. We use a Long Short Term Memory (LSTM) based recurrent neural network sequence model trained on dyadic conversations from psychotherapy sessions and our model outperforms a standard baseline with linguistic humor features.

Impact of Primary Care Intensive Management on High-Risk Veterans' Costs and Utilization: A Randomized Quality Improvement Trial

BACKGROUND

Primary care models that offer comprehensive, accessible care to all patients may provide insufficient resources to meet the needs of patients with complex conditions who have the greatest risk for hospitalization.

OBJECTIVE

To assess whether augmenting usual primary care with team-based intensive management lowers utilization and costs for high-risk patients.

DESIGN

Randomized quality improvement trial. (ClinicalTrials.gov: NCT03100526).

SETTING

5 U.S. Department of Veterans Affairs (VA) medical centers.

PATIENTS

Primary care patients at high risk for hospitalization who had a recent acute care episode.

INTERVENTION

Locally tailored intensive management programs providing care coordination, goals assessment, health coaching, medication reconciliation, and home visits through an interdisciplinary team, including a physician or nurse practitioner, a nurse, and psychosocial experts.

MEASUREMENTS

Utilization and costs (including intensive management program expenses) 12 months before and after randomization.

RESULTS

2210 patients were randomly assigned, 1105 to intensive management and 1105 to usual care. Patients had a mean age of 63 years and an average of 7 chronic conditions; 90% were men. Of the patients assigned to intensive management, 487 (44%) received intensive outpatient care (that is, ≥3 encounters in person or by telephone) and 204 (18%) received limited intervention. From the pre- to postrandomization periods, mean inpatient costs decreased more for the intensive management than the usual care group (-$2164 [95% CI, -$7916 to $3587]). Outpatient costs increased more for the intensive management than the usual care group ($2636 [CI, $524 to $4748]), driven by greater use of primary care, home care, telephone care, and telehealth. Mean total costs were similar in the 2 groups before and after randomization.

LIMITATIONS

Sites took up to several months to contact eligible patients, limiting the time between treatment and outcome assessment. Only VA costs were assessed.

CONCLUSION

High-risk patients with access to an intensive management program received more outpatient care with no increase in total costs.

PRIMARY FUNDING SOURCE

Veterans Health Administration Primary Care Services.

An operations-partnered evaluation of care redesign for high-risk patients in the Veterans Health Administration (VHA): Study protocol for the PACT Intensive Management (PIM) randomized quality improvement evaluation

BACKGROUND

Patient-centered medical homes have made great strides providing comprehensive care for patients with chronic conditions, but may not provide sufficient support for patients at highest risk for acute care use. To address this, the Veterans Health Administration (VHA) initiated a five-site demonstration project to evaluate the effectiveness of augmenting the VA's Patient Aligned Care Team (PACT) medical home with PACT Intensive Management (PIM) teams for Veterans at highest risk for hospitalization.

METHODS/DESIGN

Researchers partnered with VHA leadership to design a mixed-methods prospective multi-site evaluation that met leadership's desire for a rigorous evaluation conducted as quality improvement rather than research. We conducted a randomized QI evaluation and assigned high-risk patients to participate in PIM and compared them with high-risk Veterans receiving usual care through PACT. The summative evaluation examines whether PIM: 1) decreases VHA emergency department and hospital use; 2) increases satisfaction with VHA care; 3) decreases provider burnout; and 4) generates positive returns on investment. The formative evaluation aims to support improved care for high-risk patients at demonstration sites and to inform future initiatives for high-risk patients. The evaluation was reviewed by representatives from the VHA Office of Research and Development and the Office of Research Oversight and met criteria for quality improvement.

DISCUSSION

VHA aims to function as a learning organization by rapidly implementing and rigorously testing QI innovations prior to final program or policy development. We observed challenges and opportunities in designing an evaluation consistent with QI standards and operations priorities, while also maintaining scientific rigor.

TRIAL REGISTRATION

This trial was retrospectively registered at ClinicalTrials.gov on April 3, 2017: NCT03100526. Protocol v1, FY14-17.

Moving From Discovery to System-Wide Change: The Role of Research in a Learning Health Care System: Experience from Three Decades of Health Systems Research in the Veterans Health Administration

The Veterans Health Administration is unique, functioning as an integrated health care system that provides care to more than six million veterans annually and as a home to an established scientific enterprise that conducts more than $1 billion of research each year. The presence of research, spanning the continuum from basic health services to translational research, has helped the Department of Veterans Affairs (VA) realize the potential of a learning health care system and has contributed to significant improvements in clinical quality over the past two decades. It has also illustrated distinct pathways by which research influences clinical care and policy and has provided lessons on challenges in translating research into practice on a national scale. These lessons are increasingly relevant to other health care systems, as the issues confronting the VA—the need to provide timely access, coordination of care, and consistent high quality across a diverse system—mirror those of the larger US health care system.

Proceedings of the 8th Annual Conference on the Science of Dissemination and Implementation : Washington, DC, USA. 14-15 December 2015

A1 Introduction to the 8^th Annual Conference on the Science of Dissemination and Implementation: Optimizing Personal and Population Health

David Chambers, Lisa Simpson

D1 Discussion forum: Population health D&I research

Felicia Hill-Briggs

D2 Discussion forum: Global health D&I research

Gila Neta, Cynthia Vinson

D3 Discussion forum: Precision medicine and D&I research

David Chambers

S1 Predictors of community therapists’ use of therapy techniques in a large public mental health system

Rinad Beidas, Steven Marcus, Gregory Aarons, Kimberly Hoagwood, Sonja Schoenwald, Arthur Evans, Matthew Hurford, Ronnie Rubin, Trevor Hadley, Frances Barg, Lucia Walsh, Danielle Adams, David Mandell

S2 Implementing brief cognitive behavioral therapy (CBT) in primary care: Clinicians' experiences from the field

Lindsey Martin, Joseph Mignogna, Juliette Mott, Natalie Hundt, Michael Kauth, Mark Kunik, Aanand Naik, Jeffrey Cully

S3 Clinician competence: Natural variation, factors affecting, and effect on patient outcomes

Alan McGuire, Dominique White, Tom Bartholomew, John McGrew, Lauren Luther, Angie Rollins, Michelle Salyers

S4 Exploring the multifaceted nature of sustainability in community-based prevention: A mixed-method approach

Brittany Cooper, Angie Funaiole

S5 Theory informed behavioral health integration in primary care: Mixed methods evaluation of the implementation of routine depression and alcohol screening and assessment

Julie Richards, Amy Lee, Gwen Lapham, Ryan Caldeiro, Paula Lozano, Tory Gildred, Carol Achtmeyer, Evette Ludman, Megan Addis, Larry Marx, Katharine Bradley

S6 Enhancing the evidence for specialty mental health probation through a hybrid efficacy and implementation study

Tonya VanDeinse, Amy Blank Wilson, Burgin Stacey, Byron Powell, Alicia Bunger, Gary Cuddeback

S7 Personalizing evidence-based child mental health care within a fiscally mandated policy reform

Miya Barnett, Nicole Stadnick, Lauren Brookman-Frazee, Anna Lau

S8 Leveraging an existing resource for technical assistance: Community-based supervisors in public mental health

Shannon Dorsey, Michael Pullmann

S9 SBIRT implementation for adolescents in urban federally qualified health centers: Implementation outcomes

Shannon Mitchell, Robert Schwartz, Arethusa Kirk, Kristi Dusek, Marla Oros, Colleen Hosler, Jan Gryczynski, Carolina Barbosa, Laura Dunlap, David Lounsbury, Kevin O'Grady, Barry Brown

S10 PANEL: Tailoring Implementation Strategies to Context - Expert recommendations for tailoring strategies to context

Laura Damschroder, Thomas Waltz, Byron Powell

S11 PANEL: Tailoring Implementation Strategies to Context - Extreme facilitation: Helping challenged healthcare settings implement complex programs

Mona Ritchie

S12 PANEL: Tailoring Implementation Strategies to Context - Using menu-based choice tasks to obtain expert recommendations for implementing three high-priority practices in the VA

Thomas Waltz

S13 PANEL: The Use of Technology to Improve Efficient Monitoring of Implementation of Evidence-based Programs - Siri, rate my therapist: Using technology to automate fidelity ratings of motivational interviewing

David Atkins, Zac E. Imel, Bo Xiao, Doğan Can, Panayiotis Georgiou, Shrikanth Narayanan

S14 PANEL: The Use of Technology to Improve Efficient Monitoring of Implementation of Evidence-based Programs - Identifying indicators of implementation quality for computer-based ratings

Cady Berkel, Carlos Gallo, Irwin Sandler, C. Hendricks Brown, Sharlene Wolchik, Anne Marie Mauricio

S15 PANEL: The Use of Technology to Improve Efficient Monitoring of Implementation of Evidence-based Programs - Improving implementation of behavioral interventions by monitoring emotion in spoken speech

Carlos Gallo, C. Hendricks Brown, Sanjay Mehrotra

S16 Scorecards and dashboards to assure data quality of health management information system (HMIS) using R

Dharmendra Chandurkar, Siddhartha Bora, Arup Das, Anand Tripathi, Niranjan Saggurti, Anita Raj

S17 A big data approach for discovering and implementing patient safety insights

Eric Hughes, Brian Jacobs, Eric Kirkendall

S18 Improving the efficacy of a depression registry for use in a collaborative care model

Danielle Loeb, Katy Trinkley, Michael Yang, Andrew Sprowell, Donald Nease

S19 Measurement feedback systems as a strategy to support implementation of measurement-based care in behavioral health

Aaron Lyon, Cara Lewis, Meredith Boyd, Abigail Melvin, Semret Nicodimos, Freda Liu, Nathanial Jungbluth

S20 PANEL: Implementation Science and Learning Health Systems: Intersections and Commonalities - Common loop assay: Methods of supporting learning collaboratives

Allen Flynn

S21 PANEL: Implementation Science and Learning Health Systems: Intersections and Commonalities - Innovating audit and feedback using message tailoring models for learning health systems

Zach Landis-Lewis

S22 PANEL: Implementation Science and Learning Health Systems: Intersections and Commonalities - Implementation science and learning health systems: Connecting the dots

Anne Sales

S23 Facilitation activities of Critical Access Hospitals during TeamSTEPPS implementation

Jure Baloh, Marcia Ward, Xi Zhu

S24 Organizational and social context of federally qualified health centers and variation in maternal depression outcomes

Ian Bennett, Jurgen Unutzer, Johnny Mao, Enola Proctor, Mindy Vredevoogd, Ya-Fen Chan, Nathaniel Williams, Phillip Green

S25 Decision support to enhance treatment of hospitalized smokers: A randomized trial

Steven Bernstein, June-Marie Rosner, Michelle DeWitt, Jeanette Tetrault, James Dziura, Allen Hsiao, Scott Sussman, Patrick O’Connor, Benjamin Toll

S26 PANEL: Developing Sustainable Strategies for the Implementation of Patient-Centered Care across Diverse US Healthcare Systems - A patient-centered approach to successful community transition after catastrophic injury

Michael Jones, Julie Gassaway

S27 PANEL: Developing Sustainable Strategies for the Implementation of Patient-Centered Care across Diverse US Healthcare Systems - Conducting PCOR to integrate mental health and cancer screening services in primary care

Jonathan Tobin

S28 PANEL: Developing Sustainable Strategies for the Implementation of Patient-Centered Care across Diverse US Healthcare Systems - A comparative effectiveness trial of optimal patient-centered care for US trauma care systems

Douglas Zatzick

S29 Preferences for in-person communication among patients in a multi-center randomized study of in-person versus telephone communication of genetic test results for cancer susceptibility

Angela R Bradbury, Linda Patrick-Miller, Brian Egleston, Olufunmilayo I Olopade, Michael J Hall, Mary B Daly, Linda Fleisher, Generosa Grana, Pamela Ganschow, Dominique Fetzer, Amanda Brandt, Dana Farengo-Clark, Andrea Forman, Rikki S Gaber, Cassandra Gulden, Janice Horte, Jessica Long, Rachelle Lorenz Chambers, Terra Lucas, Shreshtha Madaan, Kristin Mattie, Danielle McKenna, Susan Montgomery, Sarah Nielsen, Jacquelyn Powers, Kim Rainey, Christina Rybak, Michelle Savage, Christina Seelaus, Jessica Stoll, Jill Stopfer, Shirley Yao and Susan Domchek

S30 Working towards de-implementation: A mixed methods study in breast cancer surveillance care

Erin Hahn, Corrine Munoz-Plaza, Jianjin Wang, Jazmine Garcia Delgadillo, Brian Mittman Michael Gould

S31Integrating evidence-based practices for increasing cancer screenings in safety-net primary care systems: A multiple case study using the consolidated framework for implementation research

Shuting (Lily) Liang, Michelle C. Kegler, Megan Cotter, Emily Phillips, April Hermstad, Rentonia Morton, Derrick Beasley, Jeremy Martinez, Kara Riehman

S32 Observations from implementing an mHealth intervention in an FQHC

David Gustafson, Lisa Marsch, Louise Mares, Andrew Quanbeck, Fiona McTavish, Helene McDowell, Randall Brown, Chantelle Thomas, Joseph Glass, Joseph Isham, Dhavan Shah

S33 A multicomponent intervention to improve primary care provider adherence to chronic opioid therapy guidelines and reduce opioid misuse: A cluster randomized controlled trial protocol

Jane Liebschutz, Karen Lasser

S34 Implementing collaborative care for substance use disorders in primary care: Preliminary findings from the summit study

Katherine Watkins, Allison Ober, Sarah Hunter, Karen Lamp, Brett Ewing

S35 Sustaining a task-shifting strategy for blood pressure control in Ghana: A stakeholder analysis

Juliet Iwelunmor, Joyce Gyamfi, Sarah Blackstone, Nana Kofi Quakyi, Jacob Plange-Rhule, Gbenga Ogedegbe

S36 Contextual adaptation of the consolidated framework for implementation research (CFIR) in a tobacco cessation study in Vietnam

Pritika Kumar, Nancy Van Devanter, Nam Nguyen, Linh Nguyen, Trang Nguyen, Nguyet Phuong, Donna Shelley

S37 Evidence check: A knowledge brokering approach to systematic reviews for policy

Sian Rudge

S38 Using Evidence Synthesis to Strengthen Complex Health Systems in Low- and Middle-Income Countries

Etienne Langlois

S39 Does it matter: timeliness or accuracy of results? The choice of rapid reviews or systematic reviews to inform decision-making

Andrea Tricco

S40 Evaluation of the veterans choice program using lean six sigma at a VA medical center to identify benefits and overcome obstacles

Sherry Ball, Anne Lambert-Kerzner, Christine Sulc, Carol Simmons, Jeneen Shell-Boyd, Taryn Oestreich, Ashley O'Connor, Emily Neely, Marina McCreight, Amy Labebue, Doreen DiFiore, Diana Brostow, P. Michael Ho, David Aron

S41 The influence of local context on multi-stakeholder alliance quality improvement activities: A multiple case study

Jillian Harvey, Megan McHugh, Dennis Scanlon

S42 Increasing physical activity in early care and education: Sustainability via active garden education (SAGE)

Rebecca Lee, Erica Soltero, Nathan Parker, Lorna McNeill, Tracey Ledoux

S43 Marking a decade of policy implementation: The successes and continuing challenges of a provincial school food and nutrition policy in Canada

Jessie-Lee McIsaac, Kate MacLeod, Nicole Ata, Sherry Jarvis, Sara Kirk

S44 Use of research evidence among state legislators who prioritize mental health and substance abuse issues

Jonathan Purtle, Elizabeth Dodson, Ross Brownson

S45 PANEL: Effectiveness-Implementation Hybrid Designs: Clarifications, Refinements, and Additional Guidance Based on a Systematic Review and Reports from the Field - Hybrid type 1 designs

Brian Mittman, Geoffrey Curran

S46 PANEL: Effectiveness-Implementation Hybrid Designs: Clarifications, Refinements, and Additional Guidance Based on a Systematic Review and Reports from the Field - Hybrid type 2 designs

Geoffrey Curran

S47 PANEL: Effectiveness-Implementation Hybrid Designs: Clarifications, Refinements, and Additional Guidance Based on a Systematic Review and Reports from the Field - Hybrid type 3 designs

Jeffrey Pyne

S48 Linking team level implementation leadership and implementation climate to individual level attitudes, behaviors, and implementation outcomes

Gregory Aarons, Mark Ehrhart, Elisa Torres

S49 Pinpointing the specific elements of local context that matter most to implementation outcomes: Findings from qualitative comparative analysis in the RE-inspire study of VA acute stroke care

Edward Miech

S50 The GO score: A new context-sensitive instrument to measure group organization level for providing and improving care

Edward Miech

S51 A research network approach for boosting implementation and improvement

Kathleen Stevens, I.S.R.N. Steering Council

S52 PANEL: Qualitative methods in D&I Research: Value, rigor and challenge - The value of qualitative methods in implementation research

Alison Hamilton

S53 PANEL: Qualitative methods in D&I Research: Value, rigor and challenge - Learning evaluation: The role of qualitative methods in dissemination and implementation research

Deborah Cohen

S54 PANEL: Qualitative methods in D&I Research: Value, rigor and challenge - Qualitative methods in D&I research

Deborah Padgett

S55 PANEL: Maps & models: The promise of network science for clinical D&I - Hospital network of sharing patients with acute and chronic diseases in California

Alexandra Morshed

S56 PANEL: Maps & models: The promise of network science for clinical D&I - The use of social network analysis to identify dissemination targets and enhance D&I research study recruitment for pre-exposure prophylaxis for HIV (PrEP) among men who have sex with men

Rupa Patel

S57 PANEL: Maps & models: The promise of network science for clinical D&I - Network and organizational factors related to the adoption of patient navigation services among rural breast cancer care providers

Beth Prusaczyk

S58 A theory of de-implementation based on the theory of healthcare professionals’ behavior and intention (THPBI) and the becker model of unlearning

David C. Aron, Divya Gupta, Sherry Ball

S59 Observation of registered dietitian nutritionist-patient encounters by dietetic interns highlights low awareness and implementation of evidence-based nutrition practice guidelines

Rosa Hand, Jenica Abram, Taylor Wolfram

S60 Program sustainability action planning: Building capacity for program sustainability using the program sustainability assessment tool

Molly Hastings, Sarah Moreland-Russell

S61 A review of D&I study designs in published study protocols

Rachel Tabak, Alex Ramsey, Ana Baumann, Emily Kryzer, Katherine Montgomery, Ericka Lewis, Margaret Padek, Byron Powell, Ross Brownson

S62 PANEL: Geographic variation in the implementation of public health services: Economic, organizational, and network determinants - Model simulation techniques to estimate the cost of implementing foundational public health services

Cezar Brian Mamaril, Glen Mays, Keith Branham, Lava Timsina

S63 PANEL: Geographic variation in the implementation of public health services: Economic, organizational, and network determinants - Inter-organizational network effects on the implementation of public health services

Glen Mays, Rachel Hogg

S64 PANEL: Building capacity for implementation and dissemination of the communities that care prevention system at scale to promote evidence-based practices in behavioral health - Implementation fidelity, coalition functioning, and community prevention system transformation using communities that care

Abigail Fagan, Valerie Shapiro, Eric Brown

S65 PANEL: Building capacity for implementation and dissemination of the communities that care prevention system at scale to promote evidence-based practices in behavioral health - Expanding capacity for implementation of communities that care at scale using a web-based, video-assisted training system

Kevin Haggerty, David Hawkins

S66 PANEL: Building capacity for implementation and dissemination of the communities that care prevention system at scale to promote evidence-based practices in behavioral health - Effects of communities that care on reducing youth behavioral health problems

Sabrina Oesterle, David Hawkins, Richard Catalano

S68 When interventions end: the dynamics of intervention de-adoption and replacement

Virginia McKay, M. Margaret Dolcini, Lee Hoffer

S69 Results from next-d: can a disease specific health plan reduce incident diabetes development among a national sample of working-age adults with pre-diabetes?

Tannaz Moin, Jinnan Li, O. Kenrik Duru, Susan Ettner, Norman Turk, Charles Chan, Abigail Keckhafer, Robert Luchs, Sam Ho, Carol Mangione

S70 Implementing smoking cessation interventions in primary care settings (STOP): using the interactive systems framework

Peter Selby, Laurie Zawertailo, Nadia Minian, Dolly Balliunas, Rosa Dragonetti, Sarwar Hussain, Julia Lecce

S71 Testing the Getting To Outcomes implementation support intervention in prevention-oriented, community-based settings

Matthew Chinman, Joie Acosta, Patricia Ebener, Patrick S Malone, Mary Slaughter

S72 Examining the reach of a multi-component farmers’ market implementation approach among low-income consumers in an urban context

Darcy Freedman, Susan Flocke, Eunlye Lee, Kristen Matlack, Erika Trapl, Punam Ohri-Vachaspati, Morgan Taggart, Elaine Borawski

S73 Increasing implementation of evidence-based health promotion practices at large workplaces: The CEOs Challenge

Amanda Parrish, Jeffrey Harris, Marlana Kohn, Kristen Hammerback, Becca McMillan, Peggy Hannon

S74 A qualitative assessment of barriers to nutrition promotion and obesity prevention in childcare

Taren Swindle, Geoffrey Curran, Leanne Whiteside-Mansell, Wendy Ward

S75 Documenting institutionalization of a health communication intervention in African American churches

Cheryl Holt, Sheri Lou Santos, Erin Tagai, Mary Ann Scheirer, Roxanne Carter, Janice Bowie, Muhiuddin Haider, Jimmie Slade, Min Qi Wang

S76 Reduction in hospital utilization by underserved patients through use of a community-medical home

Andrew Masica, Gerald Ogola, Candice Berryman, Kathleen Richter

S77 Sustainability of evidence-based lay health advisor programs in African American communities: A mixed methods investigation of the National Witness Project

Rachel Shelton, Lina Jandorf, Deborah Erwin

S78 Predicting the long-term uninsured population and analyzing their gaps in physical access to healthcare in South Carolina

Khoa Truong

S79 Using an evidence-based parenting intervention in churches to prevent behavioral problems among Filipino youth: A randomized pilot study

Joyce R. Javier, Dean Coffey, Sheree M. Schrager, Lawrence Palinkas, Jeanne Miranda

S80 Sustainability of elementary school-based health centers in three health-disparate southern communities

Veda Johnson, Valerie Hutcherson, Ruth Ellis

S81 Childhood obesity prevention partnership in Louisville: creative opportunities to engage families in a multifaceted approach to obesity prevention

Anna Kharmats, Sandra Marshall-King, Monica LaPradd, Fannie Fonseca-Becker

S82 Improvements in cervical cancer prevention found after implementation of evidence-based Latina prevention care management program

Deanna Kepka, Julia Bodson, Echo Warner, Brynn Fowler

S83 The OneFlorida data trust: Achieving health equity through research & training capacity building

Elizabeth Shenkman, William Hogan, Folakami Odedina, Jessica De Leon, Monica Hooper, Olveen Carrasquillo, Renee Reams, Myra Hurt, Steven Smith, Jose Szapocznik, David Nelson, Prabir Mandal

S84 Disseminating and sustaining medical-legal partnerships: Shared value and social return on investment

James Teufel

Patterns of adaptation to children's food allergies

BACKGROUND

Families with food allergy (FA) are at risk of reduced quality of life and elevated anxiety. A moderate level of anxiety may be beneficial to sustain vigilance for food avoidance; however, excessive anxiety may increase risk for burden and maladjustment. The current study presents a framework for understanding the patterns of adaptation to FA across families and to identify typologies of families that would benefit from intervention.

METHODS

Participants included 57 children, 6-12 years old with documented FA, and their mothers. Families were assessed using the Food Allergy Management and Adaptation Scale. Families also completed measures of quality of life, anxiety, FA management, and psychosocial impairment.

RESULTS

A hierarchical cluster analysis revealed that 56 of the 57 families of food-allergic children were categorized into four groups that differed on their adequacy of family FA management, levels of anxiety, and balanced psychosocial functioning: balanced responders (n = 23; 41%), high responders (n = 25; 45%), and low responders (n = 3; 5%). The fourth group, anxious high responders (n = 5; 9%), was characterized by extremely high maternal FA anxiety scores and low scores for balanced integration of FA management and psychosocial functioning. Families in clusters differed across illness and psychosocial outcome variables.

CONCLUSION

Families with FA were characterized by patterns of FA management, anxiety, and ability to integrate FA demands into daily life. Identified adaptation patterns correspond with clinical impressions and provide a framework for identifying families in need of intervention.

Evidence-Based Policy Making: Balancing Rigor With Real-World Health Care for Veterans and Military Personnel

The US health care system is undergoing unprecedented policy transformations that will impact veterans and returning service members. Rigorous program evaluations will be crucial as policy makers make tough decisions regarding the value of new policies. Also, more partnerships between scientists and policy makers will be required to implement these designs in light of trade-offs between scientific rigor and political realities.

Culture shapes empathic responses to physical and social pain

The present research investigates the extent to which cultural background moderates empathy in response to observing someone undergoing physical or social pain. In 3 studies, we demonstrate that, East Asian and White British participants differ in both affective and cognitive components of their empathic reactions in response to someone else’s pain. Compared with East Asian participants, British participants report greater empathic concern and show lower empathic accuracy. More important, findings cannot be explained by an in-group advantage effect. Potential reasons for observed cultural differences are discussed.

Drinking norms, readiness to change, and gender as moderators of a combined alcohol intervention for first-year college students

INTRODUCTION

Alcohol interventions targeting college students and their parents have been shown to be efficacious. Little research has examined moderators of intervention efficacy to help tailor interventions for subgroups of students.

METHOD

This study is a secondary data analysis of readiness to change, drinking norms, and gender as moderators of an efficacious peer- and parent-based intervention (Turrisi et al., 2009). Students (n=680) were randomized to the combined peer and parent intervention (n=342) or assessment-only control (n=338).

RESULTS

The combined intervention reduced peak blood alcohol content (BAC) compared to control. Gender and norms did not moderate the relationship between the intervention and drinking. Significant interactions were found between gender, precontemplation, and intervention. Students in the combined condition with higher precontemplation had lower weekly drinking compared to those with lower precontemplation. This pattern was also found among men for peak BAC and alcohol-related consequences but not among women, indicating a three-way interaction.

CONCLUSION

Interventions may need to consider readiness to change and gender to optimize effectiveness.

A Randomized Evaluation of Motivational Interviewing Training for Mandated Implementation of Alcohol Screening and Brief Intervention in Trauma Centers

The American College of Surgeons has mandated that level I and level II trauma centers implement universal alcohol screening and brief intervention (SBI) for injured patients. This study was a secondary analysis of a national, 20-hospital, cluster-randomized implementation trial focusing on practical issues of training and supervising alcohol SBI providers in motivational interviewing (MI). The purpose of this study was to examine whether real-world trauma center providers can be trained to provide higher quality counseling using MI as part of brief interventions for alcohol and whether MI skills can be maintained over time. Sites were randomly assigned to receive a 1day workshop training in MI for alcohol SBI or not, and all providers regardless of training completed up to seven standardized patient assessments of MI fidelity over 27months. Six domains on the Motivational Interviewing Treatment Integrity (MITI) coding system were assessed and compared to proficiency criteria. Providers in the intervention training group showed substantially improved MITI scores over the course of the 27-month time period. Domains that had particularly strong improvement were MI spirit and empathy; however, despite the overall improvement in the intervention group scores, expert-derived proficiency criteria were attained only for the global scores. Routine trauma center providers who receive MI training can deliver higher quality counseling in alcohol brief interventions, but may not, however, attain previously derived proficiency standards. Future implementation efforts in real-world acute care medical settings could further elucidate provider characteristics that predict training response and also strive to demonstrate that higher quality alcohol SBI implementation is associated with improved patient-level outcomes.