Shadow Coaching Improves Patient Experience With Care, But Gains Erode Later

Improving Patient Experience With Provider Communication: Systematic Review of Interventions, Implementation Strategies, and Their Effectiveness

BACKGROUND

Provider communication with patients may be improved through training, shadow coaching, and other in-service interventions. We aim to synthesize these interventions, implementation strategies, and their impact on the patient experience.

METHODS

A systematic review of contemporary evidence (2015-2023). Six scientific databases, specialty journals, and snowballing searches identified training, shadow coaching, and other in-service interventions for improving provider communication with patients, evaluated by standardized patient experience measures. Studies without inferential statistics were excluded. Two independent reviewers assessed the studies' eligibility and methodological quality and mapped the implementation strategies against a widely used taxonomy of 73 strategies.

RESULTS

Of 1237 papers screened, 14 were included: 10 controlled studies (5 randomized) and 4 prepost. Nine studies were on communication skills training and 3 on shadow coaching; all but one of these used a train-the-trainer implementation strategy. Eight studies (controlled n=4) used 5.5-8 hours of communication training and showed significant improvements in selected experience outcomes. Brief (45 min) communication training showed no significant results. Two controlled studies showed that shadow coaching and recoaching achieve short-term improvements but eroded without booster sessions. The use of transparent surgeon masks improved selected communication outcomes, but periodic reminders sent to clinicians on communication etiquette did not.

DISCUSSION

In-service communication training (≥5.5 h) or shadow (re-)coaching by trained peers can improve patients' experience with provider-patient communication. To implement such interventions, organizations need to identify and train trainers/coaches, intentionally support the program, monitor effectiveness, and add boosters as needed. Brief communication etiquette training or simple reminders did not improve the patients' experiences with provider-patient communication.

A perspective on the use of patient-reported experience and patient-reported outcome measures in ambulatory healthcare

INTRODUCTION

Patient-reported experience measures (PREMs) are patient reports about their healthcare, whereas patient-reported outcome measures (PROMs) are reports about their functioning and well-being regarding physical, mental, and social health. We provide a perspective on using PREMs and PROMs in ambulatory healthcare.

AREAS COVERED

We conducted a narrative review of the literature about using PREMs and PROMs in research and clinical practice, identified challenges and possibilities for addressing them, and provided suggestions for future research and clinical practice.

EXPERT OPINION

Substantial progress in using PREMs and PROMs has occurred during the last half-century. Collecting and reporting PREMs to clinicians in ambulatory care settings has improved communication with patients, diagnosis, and treatment, which may improve patients' health. Optimal use requires appropriate data analysis, minimizing implementation barriers, and facilitating interpretation of PREMs and PROMs in clinical practice. Also, formal structures and processes that include patient and family input into care improvement are needed (e.g. patient and family advisory councils as partners in co-design and coproduction of quality improvement). PREMs and PROMs have been used primarily in more affluent countries (e.g. the United States, Australia, United Kingdom, Netherlands, Japan, and Portugal), but this is expected to increase in many countries.

How the CAHPS Clinician and Group Patient Experience Survey Data Have Been Used in Research: A Systematic Review

Purpose

Patient experience is a key aspect of care quality. The Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS®) survey measures experiences with ambulatory care providers to inform public reporting, pay-for-performance initiatives, interventions, patient choice of physicians/practices, and quality improvement. Since the survey's 2007 release, no systematic review of its use in research has been published.

Methods

We reviewed English-language, peer-reviewed articles published since 2008 using CG-CAHPS survey data in the U.S. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and used the Checklist for Analytical Cross-Sectional Studies.

Results

We examined 126 articles and included 52. Twenty-seven articles focused on general primary care, and the others focused on ambulatory specialty care. Of the 52 studies, 37 were cross-sectional, and the majority conducted patient-level regression analysis, controlling for patient characteristics. The most-used CAHPS measures were overall provider rating and the provider communication composite. CG-CAHPS data were primarily utilized to evaluate interventions (24 studies) and examine cross-sectional associations (21 studies) of site-level (eg, organizational climate), provider-level (physician empathy), and patient-level (medication adherence) factors with patient experience. Four studies reported disparities in patient experience.

Conclusions

The widespread use of CG-CAHPS data implies the survey's value in measuring and improving care quality. Unlike facility or plan surveys, the CG-CAHPS survey was designed to allow attribution to medical groups and clinicians, which, as evidence shows, is its main strength. Policymakers, researchers, clinicians, and health care leaders can leverage CG-CAHPS data in quality improvement efforts and interventions supporting patient-centered care.

Patient Surveys Are Used Most Often to Assess Health Care Innovations, Rigorous Methods Are Less Common

The Agency of Healthcare Research and Quality Healthcare Innovations Exchange (IE) was developed to collect and report on innovative approaches to improving health care. The team reviewed 348 IE innovations including patient-reported satisfaction or experience measures. Innovations most often measured overall rating of care (61% of innovations), followed by access (52%) and provider-patient communication (12%). More than half used patient satisfaction surveys (n = 187) rather than patient experience surveys (n = 64). Innovations using patient experience surveys more often measured specific aspects of patient care, for example, access, versus a general overall rating of care. Most innovations using patient experience surveys administered nonvalidated, homegrown surveys, with few using the Agency of Healthcare Research and Quality-endorsed, psychometrically-tested CAHPS (Consumer Assessment of Healthcare Providers and Systems) survey. The most common study design was postimplementation-only (65%), highlighting that methodological rigor used to assess patient-centeredness in the IE is low. Broad use of patient experience surveys and more rigorous evaluation study designs has increased some over time but is still lacking.

Support for Use of Consumer Assessment of Healthcare Providers and Systems Communication Items Among Seriously Ill Patients

Background: High-quality doctor-patient communication is essential for patients with serious illnesses. The reliability and validity of Consumer Assessment of Healthcare Providers and Systems (CAHPS®) communication items among these patients are unknown. Methods: Five CAHPS communication items, a 4-item Advance Care Planning (ACP) engagement scale, 5-item confidence in others' knowledge of ACP medical wishes scale, and a question about confidence in filling out ACP-related medical forms were administered to 1100 patients (20% response rate) with serious illness receiving primary care at three University of California Health Systems. Results: Average age was 69 (range 22-102); 52% male, 18% Hispanic, 9% Asian, and 7% Black; 24% had high school or less education. Eigenvalues and internal consistency reliability (0.88) supported a 5-item communication scale. Item characteristic curves showed a monotonic relationship of response options with the communication score. Item thresholds indicated that most patients reported positive patient experiences (i.e., items were negatively skewed). Item slopes ranging from 2.52 to 5.10 confirmed that all items were strongly related to the communication score. Information (reliability) of the communication scale was higher for assessing patients with negative experiences of care than for the positive end of the spectrum. Communication was positively correlated with confidence in other's knowledge of ACP medical wishes (r = 0.32, p < 0.0001), ACP engagement (r = 0.14, p < 0.0001), and confidence in filling out ACP-related medical forms (r = 0.09, p = 0.0022). Conclusions: These findings support the use of CAHPS survey items to assess communication among patients with serious illnesses in primary care. Clinical Trial Registration: https://clinicaltrials.gov/ct2/show/NCT04012749.

Shadow Coaching Improves Patient Experience for English-Preferring Patients but not for Spanish-Preferring Patients

BACKGROUND

Shadow coaching, a type of one-on-one provider counseling by trained peers, is an effective strategy for improving provider behaviors and patient interactions, but its effects on improving patient experience for English- and Spanish-preferring patients is unknown.

OBJECTIVE

Assess effects of shadow coaching on patient experience for English- and for Spanish-preferring patients.

DESIGN

We analyzed 2012-2019 Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) data (n=46,089) from an urban Federally Qualified Health Center with 44 primary care practices and 320 providers. One-third (n=14,631) were Spanish-preferring patients. We fit mixed-effects regression models with random effects for provider (the level of treatment assignment) and fixed effects for time (a linear spline for time with a knot and "jump" at coaching date), patient characteristics, and site indicators, stratified by preferred language.

PARTICIPANTS

The 74 providers who had a 6-month average top-box score on the CAHPS overall provider rating below 90 (on a 100-point scale) were shadow coached. Similar percentages of English-preferring (45%) and Spanish-preferring patients (43%) were seen by coached providers.

INTERVENTION

Trained providers observed patient care by colleagues and provided suggestions for improvement. Verbal feedback was provided immediately after the observation and the participant received a written report summarizing the comments and recommendations from the coaching session.

MAIN MEASURES

CG-CAHPS Visit Survey 2.0 provider communication composite and overall provider rating (0-100 scoring).

KEY RESULTS

We found a statistically significant 2-point (small) jump in CAHPS provider communication and overall provider rating among English-preferring patients of coached providers. There was no evidence of a coaching effect on patient experience for Spanish-preferring patients.

CONCLUSIONS

Coaching improved care experiences for English-preferring patients but may not have improved patient experience for Spanish-preferring patients. Selection and training of providers to communicate effectively with Spanish-preferring patients is needed to extend the benefits of shadow coaching to Spanish-preferring patients.

Patient Comments on the Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS) Survey Reflect Improvements in Provider Behaviors From Coaching

GOAL

Patient experience survey data are used to examine the patient-centeredness of care, identify areas for improvement, and monitor interventions aimed to enhance the patient experience. Most healthcare organizations measure patient experience using Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys. Studies have documented the use of CAHPS closed-ended survey responses for completing public reports, monitoring internal feedback and performance, identifying areas of improvement, and evaluating interventions to improve care. However, limited evidence exists on the utility of patients' comments on CAHPS surveys for evaluating provider-level interventions. To explore this potential, we examined comments on the CAHPS Clinician and Group (CG-CAHPS) 2.0 visit survey before and after a provider intervention. The "shadow coaching" intervention had been shown to improve provider performance and patient experience scores on the CG-CAHPS overall provider rating and provider communication composite.

METHODS

We examined how patient comments on the CG-CAHPS survey differed before and after shadow coaching of 74 providers. We described the valence (tone), content, and actionability of 1,935 comments-1,051 collected before coaching and 884 collected after coaching-to see how these aspects changed before and after providers were coached.

PRINCIPAL FINDINGS

Patient comments reflected improved CG-CAHPS scores after shadow coaching. The proportion of positive comments increased, and comments about doctors were more positive. Comments about time spent in the examination room decreased, apparently reflecting the decreased proportion of negative comments after coaching. Comments regarding three of the four aspects of provider communication asked on the CG-CAHPS survey were more positive after coaching (provider listens carefully, shows respect, spends enough time); the valence of comments about the fourth aspect (provider explains things in a way that is easy to understand) did not change. Also, comments describing an overall positive evaluation of the practice increased. Comments were generally less actionable after coaching, perhaps reflecting the increased positivity of the comments.

PRACTICAL APPLICATIONS

Patient comments collected before the provider intervention reflected overall improvements in provider behavior, as indicated by medium-to-large statistically significant improvements in CG-CAHPS composite scores. These results suggest that patient comments from the CG-CAHPS survey can be used as input for quality improvement or an evaluation of provider-level interventions. Tracking the valence and content of comments about providers before and after an intervention to improve care is a practical method to learn how provider behavior changes.

Follow-Up Shadow Coaching Improves Primary Care Provider-Patient Interactions and Maintains Improvements When Conducted Regularly: A Spline Model Analysis

INTRODUCTION

Shadow coaching improves provider-patient interactions, as measured by CG-CAHPS® overall provider rating (OPR) and provider communication (PC). However, these improvements erode over time.

AIM

Examine whether a second coaching session (re-coaching) improves and sustains patient experience.

SETTING

Large, urban Federally Qualified Health Center PROGRAM: Trained providers observed patient care by colleagues and provided suggestions for improvement. Providers with OPRs<90 (0-100-point scale) were eligible.

EVALUATION

We used stratified randomization based on provider type and OPR to assign half of the 40 eligible providers to re-coaching. For OPR and PC, we fit mixed-effects regression models with random-effects for provider (level of treatment assignment) and fixed-effects for time (linear spline with knots and possible "jump" at initial coaching and re-coaching), previous OPR, patient characteristics, and sites. We observed a statistically significant medium jump among re-coached providers after re-coaching on OPR (3.7 points) and PC (3.5 points); differences of 1, 3, and ≥5-points for CAHPS measures are considered small, medium, and large. Improvements from re-coaching persisted for 12 months for OPR and 8 months for PC.

DISCUSSION

Re-coaching improved patient experience more than initial coaching, suggesting the reactivation of knowledge from initial coaching. However, re-coaching gains also eroded. Coaching should occur every 6 to 12 months to maintain behaviors and scores.

Summary of the 2020 AHRQ research meeting on 'advancing methods of implementing and evaluating patient experience improvement using consumer assessment of healthcare providers and systems (CAHPS®) surveys'

BACKGROUND

The Agency for Healthcare Research and Quality held a research meeting on using Consumer Assessment of Healthcare Providers and Systems (CAHPS®) data for quality improvement (QI) and evaluating such efforts.

TOPICS COVERED.

Meeting addressed: 1)What has been learned about organizational factors/environment needed to improve patient experience? 2)How have organizations used data to improve patient experience? 3)What can evaluations using CAHPS data teach us about implementing successful programs to improve patient experience?

KEY THEMES

Providers and stakeholders need to be engaged early and often, standardize QI processes, complement CAHPS data with other data, and compile dashboards of CAHPS scores to identify and track improvement. Rigorous study designs are valuable, but much can be learned and accomplished through practical organization-level studies.

Content and Actionability of Recommendations to Providers After Shadow Coaching

BACKGROUND AND OBJECTIVES

Health care organizations track patient experience data, identify areas of improvement, monitor provider performance, and assist providers in improving their interactions with patients. Some practices use one-on-one provider counseling ("shadow coaching") to identify and modify provider behaviors. A recent evaluation of a large shadow coaching program found statistically significant improvements in coached providers' patient experience scores immediately after being coached. This study aimed to examine the content of the recommendations given to those providers aimed at improving provider-patient interactions, characterize these recommendations, and examine their actionability.

METHODS

Providers at a large, urban federally qualified health center were selected for coaching based on Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) patient experience scores (92 of 320 providers), shadowed by a trained peer coach for a half to full day and received recommendations on how to improve interactions with their patients. We coded 1082 recommendations found in the 92 coaching reports.

RESULTS

Reports contained an average of 12 recommendations. About half encouraged consistency of existing behaviors and half encouraged new behaviors. Most recommendations related to behaviors of the provider rather than support staff and targeted actions within the examination room rather than other spaces (eg, waiting room). The most common recommendations mapped to behavioral aspects of provider communication. Most recommendations targeted verbal rather than nonverbal communication behaviors. Most recommendations were actionable (ie, specific, descriptive), with recommendations that encouraged new behaviors being more actionable than those that encouraged existing actions.

CONCLUSIONS

Patient experience surveys are effective at identifying where improvement is needed but are not always informative enough to instruct providers on how to modify and improve their interactions with patients. Analyzing the feedback given to coached providers as part of an effective shadow-coaching program provides details about implementation on shadow-coaching feedback. Recommendations to providers aimed at improving their interactions with patients need to not only suggest the exact behaviors defined within patient experience survey items but also include recommended behaviors indirectly associated with those measured behaviors. Attention needs to be paid to supplementing patient experience data with explicit, tangible, and descriptive (ie, actionable) recommendations associated with the targeted, measured behaviors. Research is needed to understand how recommendations are put into practice by providers and what motivates and supports them to sustain changed behaviors.