Experiences and Perceptions of Healthcare Stakeholders in Disclosing Errors and Adverse Events to Historically Marginalized Patients

OBJECTIVES

We sought to assess the experiences and perceptions of healthcare stakeholders involved in the response to historically marginalized patients who have been harmed in healthcare. We investigated the challenges in disclosing errors and adverse events and the types of tools and resources that would better address the needs of historically marginalized patient populations.

METHODS

We conducted separate focus groups with two healthcare stakeholder groups: (1) frontline clinicians directly involved in the clinical care of historically marginalized patients and (2) risk and patient safety professionals involved in the hospital response to care breakdowns. We conducted an inductive analysis of the qualitative data to identify thematic clusters.

RESULTS

We interviewed 7 clinicians and 5 risk safety professionals, with a total sample size of 12 participants. Participants shared multilevel challenges in responding to historically marginalized patients after harm (system-, organizational-, and patient-level), such as fragmentation of care, lack of standardized protocols, and patient mistrust. Participants also identified their desired tools and resources for disclosure to meet the needs of historically marginalized patients, which included culturally appropriate toolkits, disclosure training, and the inclusion of multidisciplinary healthcare team members in the disclosure process.

CONCLUSIONS

Our results suggest that multiple interventions will be needed to achieve the goal of prompt disclosure of errors and adverse events across all populations engaged in health care. Future studies should investigate the perspectives of historically marginalized patients and their family members on how error and adverse event disclosure conversations should unfold.

Effects of Practicing With and Obtaining Crowdsourced Feedback From the Video-Based Communication Assessment App on Resident Physicians' Adverse Event Communication Skills: Pre-post Trial

BACKGROUND

US residents require practice and feedback to meet Accreditation Council for Graduate Medical Education mandates and patient expectations for effective communication after harmful errors. Current instructional approaches rely heavily on lectures, rarely provide individualized feedback to residents about communication skills, and may not assure that residents acquire the skills desired by patients. The Video-based Communication Assessment (VCA) app is a novel tool for simulating communication scenarios for practice and obtaining crowdsourced assessments and feedback on physicians' communication skills. We previously established that crowdsourced laypeople can reliably assess residents' error disclosure skills with the VCA app. However, its efficacy for error disclosure training has not been tested.

OBJECTIVE

We aimed to evaluate the efficacy of using VCA practice and feedback as a stand-alone intervention for the development of residents' error disclosure skills.

METHODS

We conducted a pre-post study in 2020 with pathology, obstetrics and gynecology, and internal medicine residents at an academic medical center in the United States. At baseline, residents each completed 2 specialty-specific VCA cases depicting medical errors. Audio responses were rated by at least 8 crowdsourced laypeople using 6 items on a 5-point scale. At 4 weeks, residents received numerical and written feedback derived from layperson ratings and then completed 2 additional cases. Residents were randomly assigned cases at baseline and after feedback assessments to avoid ordinal effects. Ratings were aggregated to create overall assessment scores for each resident at baseline and after feedback. Residents completed a survey of demographic characteristics. We used a 2×3 split-plot ANOVA to test the effects of time (pre-post) and specialty on communication ratings.

RESULTS

In total, 48 residents completed 2 cases at time 1, received a feedback report at 4 weeks, and completed 2 more cases. The mean ratings of residents' communication were higher at time 2 versus time 1 (3.75 vs 3.53; P<.001). Residents with prior error disclosure experience performed better at time 1 compared to those without such experience (ratings: mean 3.63 vs mean 3.46; P=.02). No differences in communication ratings based on specialty or years in training were detected. Residents' communication was rated higher for angry cases versus sad cases (mean 3.69 vs mean 3.58; P=.01). Less than half of all residents (27/62, 44%) reported prior experience with disclosing medical harm to patients; experience differed significantly among specialties (P<.001) and was lowest for pathology (1/17, 6%).

CONCLUSIONS

Residents at all training levels can potentially improve error disclosure skills with VCA practice and feedback. Error disclosure curricula should prepare residents for responding to various patient affects. Simulated error disclosure may particularly benefit trainees in diagnostic specialties, such as pathology, with infrequent real-life error disclosure practice opportunities. Future research should examine the effectiveness, feasibility, and acceptability of VCA within a longitudinal error disclosure curriculum.

Video-Based Communication Assessment of Physician Error Disclosure Skills by Crowdsourced Laypeople and Patient Advocates Who Experienced Medical Harm: Reliability Assessment With Generalizability Theory

BACKGROUND

Residents may benefit from simulated practice with personalized feedback to prepare for high-stakes disclosure conversations with patients after harmful errors and to meet American Council on Graduate Medical Education mandates. Ideally, feedback would come from patients who have experienced communication after medical harm, but medical researchers and leaders have found it difficult to reach this community, which has made this approach impractical at scale. The Video-Based Communication Assessment app is designed to engage crowdsourced laypeople to rate physician communication skills but has not been evaluated for use with medical harm scenarios.

OBJECTIVE

We aimed to compare the reliability of 2 assessment groups (crowdsourced laypeople and patient advocates) in rating physician error disclosure communication skills using the Video-Based Communication Assessment app.

METHODS

Internal medicine residents used the Video-Based Communication Assessment app; the case, which consisted of 3 sequential vignettes, depicted a delayed diagnosis of breast cancer. Panels of patient advocates who have experienced harmful medical error, either personally or through a family member, and crowdsourced laypeople used a 5-point scale to rate the residents' error disclosure communication skills (6 items) based on audiorecorded responses. Ratings were aggregated across items and vignettes to create a numerical communication score for each physician. We used analysis of variance, to compare stringency, and Pearson correlation between patient advocates and laypeople, to identify whether rank order would be preserved between groups. We used generalizability theory to examine the difference in assessment reliability between patient advocates and laypeople.

RESULTS

Internal medicine residents (n=20) used the Video-Based Communication Assessment app. All patient advocates (n=8) and 42 of 59 crowdsourced laypeople who had been recruited provided complete, high-quality ratings. Patient advocates rated communication more stringently than crowdsourced laypeople (patient advocates: mean 3.19, SD 0.55; laypeople: mean 3.55, SD 0.40; P<.001), but patient advocates' and crowdsourced laypeople's ratings of physicians were highly correlated (r=0.82, P<.001). Reliability for 8 raters and 6 vignettes was acceptable (patient advocates: G coefficient 0.82; crowdsourced laypeople: G coefficient 0.65). Decision studies estimated that 12 crowdsourced layperson raters and 9 vignettes would yield an acceptable G coefficient of 0.75.

CONCLUSIONS

Crowdsourced laypeople may represent a sustainable source of reliable assessments of physician error disclosure skills. For a simulated case involving delayed diagnosis of breast cancer, laypeople correctly identified high and low performers. However, at least 12 raters and 9 vignettes are required to ensure adequate reliability and future studies are warranted. Crowdsourced laypeople rate less stringently than raters who have experienced harm. Future research should examine the value of the Video-Based Communication Assessment app for formative assessment, summative assessment, and just-in-time coaching of error disclosure communication skills.

Disclosure Coaching: An Ask-Tell-Ask Model to Support Clinicians in Disclosure Conversations

ABSTRACT

Error disclosure is a high-stakes, emotionally charged interaction for patients and families as well as clinicians. A failed disclosure can result in emotional distress, reduced patient and family trust, litigation, and lost opportunities to learn from and prevent subsequent errors. However, many clinicians have little expertise in handling these challenging interactions and can inadvertently make a bad situation worse. Even those clinicians who have had formal disclosure training may have trouble remembering what they were taught when faced with the need to actually discuss an error with patients. Providing just-in-time coaching to clinicians is recommended by national standards. However, there is scant training material to guide error disclosure coaches. Therefore, we developed an "Ask-Tell-Ask" model and materials to guide the disclosure coaching process. The Ask-Tell-Ask model is well-suited to provide clinicians with targeted interactive teaching immediately before a disclosure without overwhelming them with lecture-style facts that they are unlikely to retain. Such teaching would ideally be provided by trained disclosure coaches, available for just-in-time support of clinicians throughout the disclosure process. The Ask-Tell-Ask model can also help risk managers, department heads, clinical managers, attending physicians, service chiefs, and others who assist clinicians with error disclosure. Here, we describe a comprehensive approach to coaching developed over years of coaching experience that incorporates the model, its rationale, step-by-step coaching strategies and guidance (including sample scripts), and organizational considerations regarding implementation of a coaching program to support patient-centered transparent communication after harmful events.

Encouraging Patients to Speak up About Problems in Cancer Care

OBJECTIVES

Many patients with cancer believe that something has gone wrong in their care but are reluctant to speak up. This pilot study sought to evaluate the impact of an intervention of active outreach to patients undergoing cancer treatment, wherein patients were encouraged to speak up if they had concerns about their care and to describe the types of concerns patients reported.

METHODS

Patients receiving cancer care at two sites were randomly assigned to an intervention or control group. Intervention patients received a brochure encouraging them to speak up about any concerns and an outreach telephone call during which the interviewer explicitly asked about concerns. Participants in both groups received baseline and follow-up questionnaires assessing their perceptions of their care and whether anything had "gone wrong" and provided ratings of health care providers' communication and responsiveness. Qualitative content coding was used to categorize patient-reported concerns collected through the baseline and follow-up questionnaires (both groups) and during telephone outreach (intervention patients only). The primary outcome was the number of patients reporting a concern about their care. Communication and responsiveness ratings for intervention and control group patients were compared using t tests.

RESULTS

Of the 60 patients in the intervention group, 34 (56.7%) reported at least one problem or concern, compared with 16 (29.1%) of the 55 patients in the control group (P = 0.003). The telephone outreach in particular resulted in more than half of those reached reporting a new concern (55.3%). We detected no impact of the intervention on patients' ratings of communication or support for speaking up.

CONCLUSIONS

Patients in this study reported a variety of concerns in response to active outreach, demonstrating that active outreach to patients can provide healthcare teams and systems the opportunity to offer a real-time response to the patient, identify where system improvements are needed, and implement policies, procedures, or programs to prevent recurrences.

Long-Term Impacts Faced by Patients and Families After Harmful Healthcare Events

BACKGROUND

Patients and families report experiencing a multitude of harms from medical errors resulting in physical, emotional, and financial hardships. Little is known about the duration and nature of these harms and the type of support needed to promote patient and family healing after such events. We sought to describe the long-term impacts (LTIs) reported by patients and family members who experienced harmful medical events 5 or more years ago.

METHODS

We performed a content analysis on 32 interviews originally conducted with 72 patients or family members about their views of the factors contributing to their self-reported harmful event. Interviews selected occurred 5 or more years after the harmful event and were grouped by time since event, 5 to 9 years (22 interviews) or 10 or more years (10 interviews) for analysis. We analyzed these interviews targeting spontaneous references of ongoing impacts experienced by the participants.

RESULTS

Participants collectively described the following four LTIs: psychological, social/behavioral, physical, and financial. Most cited psychological impacts with half-reporting ongoing anger and vivid memories. More than half reported ongoing physical impacts and one-third experienced ongoing financial impacts. Long-term social and behavioral impacts such as alterations in lifestyle, self-identity, and healthcare seeking behaviors were the most highly reported.

CONCLUSIONS

These patients and families experienced many profound LTIs after their harmful medical event. For some, these impacts evolved into secondary harms ongoing 10 years and more after the event. Our results draw attention to the persistent impacts patients and families may experience long after harmful events and the need for future research to understand and support affected patients and families.

Risk Communication After Health Care Exposures: An Experimental Vignette Survey With Patients

Purpose. We investigated how health care systems should communicate with patients about possible exposures to blood-borne pathogens that may have occurred during their care. Our goal was to determine how best to communicate uncertain risk information in a way that would minimize harm to patients, maintain their trust, and encourage patients to seek follow-up treatment. Methods. Participants (N = 1103) were randomized to receive one of six vignette surveys; 997 (98.4%) responded. All vignettes described the same event, but differed by risk level and recommendations (lower risk v. higher risk) and by communication mode (telephone, letter, social media). We measured participants’ perceived risk of blood-borne infection, trust in the health care system, and shared decision making about next clinical steps. Open-ended questions were analyzed using grounded thematic analysis. Results. When the vignette requested patients to undergo testing and practice certain health behaviors (higher risk), participants’ likelihood of seeking follow-up testing for blood-borne pathogens and their understanding of health issues increased. Perceived trust was unaffected by risk level or communication processes. Qualitative data indicated a desire for telephone communication from providers known to the patient. Limitations. It is not clear whether higher risk language or objective risk levels in vignettes motivated patients’ behavioral intentions. Conclusion. Using higher risk language when disclosing large-scale adverse events increased participants’ willingness to seek follow-up care. Implications. Health care organizations’ disclosures should focus on the next steps to take after health care exposures. This communication should involve helping patients to understand their personal health issues better, make them feel that they know which steps to take following the receipt of this information, and encouraging them to seek follow-up infectious disease testing in order to better take care of themselves.

Using crowdsourced analog patients to provide feedback on physician communication skills

OBJECTIVE

Effective physician-patient communication is important, but physicians who are seeking to improve have few opportunities for practice or receive actionable feedback. The Video-based Communication Assessment (VCA) provides both. Using the VCA, physicians respond to communication dilemmas depicted in brief video vignettes; crowdsourced analog patients rate responses and offer comments. We characterized analog patients' comments and generated actionable recommendations for improving communication.

METHODS

Physicians and residents completed the VCA; analog patients rated responses and answered:"What would you want the provider to say in this situation?" We used qualitative analysis to identify themes.

RESULTS

Forty-three participants completed the VCA; 556 analog patients provided 1035 comments. We identified overarching themes (e.g., caring, empathy, respect) and generated actionable recommendations, incorporating analog patient quotes.

CONCLUSION

While analog patients' comments could be provided directly to users, conducting a thematic analysis and developing recommendations for physician-patient communication reduced the burden on users, and allowed for focused feedback. Research is needed into physicians' reactions to the recommendations and the impact on communication.

PRACTICE IMPLICATIONS

Physicians seeking to improve communication skills may benefit from practice and feedback. The VCA was designed to provide both, incorporating the patient voice on how best to communicate in clinical situations.

Communication Training, Adverse Events, and Quality Measures: 2 Retrospective Database Analyses in Washington State Hospitals

OBJECTIVE

Washington State's HealthPact program was launched in 2011 as part of AHRQ's Patient Safety and Medical Liability Reform initiative. HealthPact delivered interdisciplinary communication training to health-care professionals with the goal of enhancing safety. We conducted 2 exploratory, retrospective database analyses to investigate training impact on the frequency of adverse events (AEs) and select quality measures across 3 time frames: pretraining (2009-2011), transition (2012), and posttraining (2013).

METHODS

Using administrative data from Washington State's Comprehensive Hospital Abstract Reporting System (CHARS) and clinical registry data from the Surgical Care and Outcomes Assessment Program (SCOAP), we compared proportions of AEs and quality measures between HealthPact (n = 4) and non-HealthPact (n = 93-CHARS; n = 48-SCOAP) participating hospitals. Risk ratios enabled comparisons between the 2 groups. Multivariable logistic regression enabled investigation of the association between training and the frequency of AEs.

RESULTS

Approximately 9.4% (CHARS) and 7.7% (SCOAP) of unique patients experienced 1 AE or greater. In CHARS, the odds of a patient experiencing an AE in a HealthPact hospital were initially (pretraining) higher than in a non-HealthPact hospital (odds ratio [OR], 1.13; 95% confidence interval [CI], 1.10-1.17), lower in transition (OR, 0.80; 95% CI, 0.76-0.83) and posttraining (OR, 0.72; 95% CI, 0.69-0.75) periods. In SCOAP, ORs were consistently lower in HealthPact hospitals: pretraining (OR, 0.87; 95% CI, 0.80-0.95), transition (OR, 0.75; 95% CI, 0.70-0.81), and posttraining (OR, 0.63; 95% CI, 0.58-0.68). The proportion of at-risk patients that experienced each individual AE was low (<1%) throughout. Adherence to quality measures was high.

CONCLUSIONS

Interprofessional communication training is an area of intense activity nationwide. A broad-based training initiative may play a role in mitigating AEs.

Communication regarding adverse neonatal birth events: Experiences of parents and clinicians

Objectives

Communicating with parents about adverse birth outcomes is challenging. We sought to describe attitudes and experiences of parents and providers regarding communication about adverse newborn birth events.

Methods

From 2011–2012, we conducted semi-structured in-depth interviews with parents who believed they had experienced an adverse birth-related neonatal outcome and focus groups with healthcare providers who have communicated with parents about adverse newborn birth events from three geographically diverse US academic medical centers. We conducted qualitative thematic analysis to identify key themes.

Results

Parents and providers described unique communication challenges around adverse neonatal outcomes in six categories: 1) High expectations for a positive delivery experience and the view that birth is a life event, not a medical encounter; 2) Powerful emotions associated with birth, amplified when an adverse event occurs; 3) Rapid changes when expectations for a normal birth take a sudden negative turn; 4) Family involvement adding complexity to communication; 5) Multiple patients and providers complicating communication dynamics with inter-professional teams seeking to coordinate information and care; and, 6) Concerns about litigation surrounding the birth experience. Strategies to educate parents and enhance communication were identified by both parents and providers.

Conclusion

Both parents and providers experience – and may suffer as a result of – communication challenges following adverse birth events affecting the newborn. Training and resources for this care environment are needed to meet parental, extended family, and provider expectations for communication when these events occur.

From implementation to sustainment: A large-scale adverse event disclosure support program generated through embedded research in the Veterans Health Administration

In 2008, the Veterans Health Administration published a groundbreaking policy on disclosing large-scale adverse events to patients in order to promote transparent communication in cases where harm may not be obvious or even certain. Without embedded research, the evidence on whether or not implementation of this policy was generating more harm than good among Veteran patients was unknown. Through an embedded research-operations partnership, we conducted four research projects that led to the development of an evidence-based large-scale disclosure toolkit and disclosure support program, and its implementation across VA healthcare. Guided by the Consolidated Framework for Implementation Research, we identified specific activities corresponding to planning, engaging, executing, reflecting and evaluating phases in the process of implementation. These activities included planning with operational leaders to establish a shared research agenda; engaging with stakeholders to discuss early results, establishing buy-in of our efforts and receiving feedback; joining existing operational teams to execute the toolkit implementation; partnering with clinical operations to evaluate the toolkit during real-time disclosures; and redesigning the toolkit to meet stakeholders' needs. Critical lessons learned for implementation success included a need for stakeholder collaboration and engagement, an organizational culture involving a strong belief in evidence, a willingness to embed researchers in clinical operation activities, allowing for testing and evaluation of innovative practices, and researchers open to constructive feedback. At the conclusion of the research, VA operations worked with the researchers to continue to support efforts to spread, scale-up and sustain toolkit use across the VA healthcare system, with the final goal to establish long-term sustainability.

Communicating with patients about diagnostic errors in breast cancer care: Providers' attitudes, experiences, and advice

OBJECTIVE

To gain understanding of breast cancer care providers' attitudes regarding communicating with patients about diagnostic errors, to inform interventions to improve patient- provider discussions.

METHODS

Focus groups were held in three U.S. states involving 41 breast cancer care providers from a variety of specialties. Discussions focused on providers' experiences with potential errors in breast cancer diagnosis, communication with patients following three hypothetical diagnostic vignettes, and suggestions for how and why diagnostic errors in breast cancer care should be communicated. Transcripts were qualitatively analyzed.

RESULTS

Providers were more willing to inform breast cancer patients of a diagnostic error when they felt it would be helpful, when they felt responsible for the error, when they were less concerned about litigation, and when the patient asked directly.

CONCLUSIONS

Breast cancer care providers experience several challenges when considering whether to inform a patient about diagnostic errors. A better understanding of patients' preferences for open communication, combined with customized tools and training, could increase clinicians' comfort with these difficult discussions.

PRACTICE IMPLICATIONS

Providers gave suggestions to facilitate discussions about diagnostic errors when these events occur, including themes of education, honesty, and optimism.

We Want to Know-A Mixed Methods Evaluation of a Comprehensive Program Designed to Detect and Address Patient-Reported Breakdowns in Care

BACKGROUND

Patients have important insights into care experiences, including breakdowns, but are often reluctant to speak up. The We Want to Know (WWTK) program was designed to make it easy for hospitalized patients to speak up about breakdowns in care and receive a response.

METHODS

The WWTK program was implemented from June 2014 through May 2017 at a large, community hospital in Baltimore. Core program features include (1) multiple channels for patients to report breakdowns, (2) campaign materials to increase patient awareness of the WWTK program, and (3) a specialist to facilitate resolution of breakdowns. This program was evaluated using mixed methods to assess the frequency and type of reported breakdowns, patient awareness of the program, and stakeholder perspectives.

RESULTS

WWTK specialists interviewed 4,676 patients; 822 (17.6%) reported a breakdown in care. Of these, 313 (38.1%) had not spoken with anyone at the hospital about the breakdown, and 547 (66.5%) described associated harm. There were also 55 patient-initiated reports to WWTK; 41 (74.5%) of these reported a care breakdown. Patients had not spoken with anyone at the hospital in 12 (29.3%) patient-initiated cases; 38 (92.7%) described associated harm. Hospital stakeholders found the level of detail and timeliness of reports to be helpful.

CONCLUSION

Active outreach to hospitalized patients detects substantially more breakdowns in care than patient-initiated reporting. Both approaches identify breakdowns that are consequential to patients and provide opportunities to respond to individual patients.

Communicating with patients about breakdowns in care: a national randomised vignette-based survey

BACKGROUND

Many patients are reluctant to speak up about breakdowns in care, resulting in missed opportunities to respond to individual patients and improve the system. Effective approaches to encouraging patients to speak up and responding when they do are needed.

OBJECTIVE

To identify factors which influence speaking up, and to examine the impact of apology when problems occur.

DESIGN

Randomised experiment using a vignette-based questionnaire describing 3 care breakdowns (slow response to call bell, rude aide, unanswered questions). The role of the person inquiring about concerns (doctor, nurse, patient care specialist), extent of the prompt (invitation to patient to share concerns) and level of apology were varied.

SETTING

National online survey.

PARTICIPANTS

1188 adults aged ≥35 years were sampled from an online panel representative of the entire US population, created and maintained by GfK, an international survey research organisation; 65.5% response rate.

MAIN OUTCOMES AND MEASURES

Affective responses to care breakdowns, intent to speak up, willingness to recommend the hospital.

RESULTS

Twice as many participants receiving an in-depth prompt about care breakdowns would (probably/definitely) recommend the hospital compared with those receiving no prompt (18.4% vs 8.8% respectively (p=0.0067)). Almost three times as many participants receiving a full apology would (probably/definitely) recommend the hospital compared with those receiving no apology (34.1% vs 13.6% respectively ((p<0.0001)). Feeling upset was a strong determinant of greater intent to speak up, but a substantial number of upset participants would not 'definitely' speak up. A more extensive prompt did not result in greater likelihood of speaking up. The inquirer's role influenced speaking up for two of the three breakdowns (rudeness and slow response).

CONCLUSIONS

Asking about possible care breakdowns in detail, and offering a full apology when breakdowns are reported substantially increases patients' willingness to recommend the hospital.

NCCN Guidelines Insights: Kidney Cancer, Version 2.2020

The NCCN Guidelines for Kidney Cancer provide multidisciplinary recommendations for the clinical management of patients with clear cell and non-clear cell renal cell carcinoma, and are intended to assist with clinical decision-making. These NCCN Guidelines Insights summarize the NCCN Kidney Cancer Panel discussions for the 2020 update to the guidelines regarding initial management and first-line systemic therapy options for patients with advanced clear cell renal cell carcinoma.

An Academic Research Coach: An Innovative Approach to Increasing Scholarly Productivity in Medicine

BACKGROUND

Academic faculty who devote most of their time to clinical work often struggle to engage in meaningful scholarly work. They may be disadvantaged by limited research training and limited time. Simply providing senior mentors and biostatistical support has limited effectiveness.

OBJECTIVE

We aimed to increase productivity in scholarly work of hospitalists and internal medicine physicians by integrating an Academic Research Coach into a robust faculty development program.

DESIGN

This was a pre-post quality improvement evaluation.

SETTING

This was conducted at the University of Washington in faculty across three academic-affiliated hospitals and 10 academic-affiliated clinics.

PARTICIPANTS

Participants were hospitalists and internists on faculty in the Division of General Internal Medicine at the University of Washington.

INTERVENTION

The coach was a 0.50 full time equivalent health services researcher with strong research methods, project implementation, and interpersonal skills. The coach consulted on research, quality improvement, and other scholarship.

MEASUREMENTS

We assessed the number of faculty supported, types of services provided, and numbers of grants, papers, and abstracts submitted and accepted.

RESULTS

The coach consulted with 49 general internal medicine faculty including 30 hospitalists who conducted 63 projects. The coach supported 13 publications, 11 abstracts, four grant submissions, and seven manuscript reviews. Forty-eight faculty in other departments benefited as co-authors.

CONCLUSION

Employing a dedicated health services researcher as part of a faculty development program is an effective way to engage clinically oriented faculty in meaningful scholarship. Key aspects of the program included an accessible and knowledgeable coach and an ongoing marketing strategy.

Capturing Emergency Department Discharge Quality With the Care Transitions Measure: A Pilot Study

BACKGROUND

Recent attention has been given to developing measures to capture the quality of ED transitions of care. We examined the utility of a patient-reported measure of transitional care, the Care Transitions Measure-3 (CTM-3), in the ED setting and its association with outcomes of care after ED discharge.

METHODS

A telephone survey was conducted of a convenience sample of patients 14 days after discharge from two emergency departments (EDs) in an academic health system. Patients responded to three statements using a four-point agreement scale (strongly disagree, disagree, agree, strongly agree): 1) "The hospital staff took my preferences and those of my family or caregiver into account when deciding what my health care needs would be"; 2) " When I left the ER, I had a good understanding of the things I was responsible for in managing my health"; and 3) "When I left the hospital, I clearly understood the purpose for taking each of my medications." Patients were also queried about outcomes after ED discharge that are known to be related to ED care transitions including medication adherence, completion of recommended follow-up, and return visits to the ED. Multivariable logistic regression was used to determine the association between the CTM-3 score (on a 100-point scale) and outcomes of interest.

RESULTS

Among 1,832 patients called, 576 were reached by phone, and 410 consented and completed our survey, representing a 22.4% response rate of patients we attempted to call. A 10-point increase in the CTM-3 score (better care experiences) was associated with a 12% decrease in the odds of having an ED return visit (adjusted odds ratio [AOR] = 0.88, 95% confidence interval [CI] = 0.77-1.00) and a 45% increase in the odds of taking prescribed medications as recommended (AOR = 1.45, 95% CI = 1.12-1.87). There was no association between CTM-3 score and completion of follow-up.

CONCLUSIONS

The CTM-3 is associated with outcomes of care after an ED visit, including ED return visits and medication adherence, and may have utility as a patient-reported measure of ED transitions of care.

Practice Implications of Expanded Genetic Testing in Oncology

Genetic test use in oncology is growing, yet providers' experiences with evolving testing norms and their implications for patient care remain under-explored. In interviews with oncologists and cancer genetics professionals, 22 key informants described the increasing importance of germline results for therapeutic decision-making, preference for ordering tests directly rather than referring, and rapid adoption of cancer gene panels for testing. Implications for informed consent, result interpretation, and patient management were identified. These results suggest concerns raised by the transition of genetic test delivery from cancer genetics professionals to oncologists that must be addressed in practice guidelines and provider training.

Twelve tips for teaching quality improvement in the clinical environment

Medical educators are expected to teach quality improvement (QI) skills alongside traditional clinical skills such as physical examination and bedside manner. Educational resources for intensive training in QI have proliferated. However, many physicians lack the time or resources to undergo this training, and may struggle with teaching these skills to their learners. In response, we offer twelve tips to help physicians teach basic QI concepts in the clinical environment. By following these tips physicians will be able to engage their learners interest in QI and provide experiential learning that makes a lasting impact.

We want to know: patient comfort speaking up about breakdowns in care and patient experience

OBJECTIVE

To assess patient comfort speaking up about problems during hospitalisation and to identify patients at increased risk of having a problem and not feeling comfortable speaking up.

DESIGN

Cross-sectional study.

SETTING

Eight hospitals in Maryland and Washington, District of Columbia.

PARTICIPANTS

Patients hospitalised at any one of eight hospitals who completed the Hospital Consumer Assessment of Healthcare Providers and Systems survey postdischarge.

MAIN OUTCOME MEASURES

Response to the question 'How often did you feel comfortable speaking up if you had any problems in your care?' grouped as: (1) no problems during hospitalisation, (2) always felt comfortable speaking up and (3) usually/sometimes/never felt comfortable speaking up.

RESULTS

Of 10 212 patients who provided valid responses, 4958 (48.6%) indicated they had experienced a problem during hospitalisation. Of these, 1514 (30.5%) did not always feel comfortable speaking up. Predictors of having a problem during hospitalisation included age, health status and education level. Patients who were older, reported worse overall and mental health, were admitted via the Emergency Department and did not speak English at home were less likely to always feel comfortable speaking up. Patients who were not always comfortable speaking up provided lower ratings of nurse communication (47.8 vs 80.4; p<0.01), physician communication (57.2 vs 82.6; p<0.01) and overall hospital ratings (7.1 vs 8.7; p<0.01). They were significantly less likely to definitely recommend the hospital (36.7% vs 71.7 %; p<0.01) than patients who were always comfortable speaking up.

CONCLUSIONS

Patients frequently experience problems in care during hospitalisation and many do not feel comfortable speaking up. Creating conditions for patients to be comfortable speaking up may result in service recovery opportunities and improved patient experience. Such efforts should consider the impact of health literacy and mental health on patient engagement in patient-safety activities.

Patient safety and the ageing physician: a qualitative study of key stakeholder attitudes and experiences

BACKGROUND

Unprecedented numbers of physicians are practicing past age 65. Unlike other safety-conscious industries, such as aviation, medicine lacks robust systems to ensure late-career physician (LCP) competence while promoting career longevity.

OBJECTIVE

To describe the attitudes of key stakeholders about the oversight of LCPs and principles that might shape policy development.

DESIGN

Thematic content analysis of interviews and focus groups.

PARTICIPANTS

40 representatives of stakeholder groups including state medical board leaders, institutional chief medical officers, senior physicians (>65 years old), patient advocates (patients or family members in advocacy roles), nurses and junior physicians. Participants represented a balanced sample from all US regions, surgical and non-surgical specialties, and both academic and non-academic institutions.

RESULTS

Stakeholders describe lax professional self-regulation of LCPs and believe this represents an important unsolved challenge. Patient safety and attention to physician well-being emerged as key organising principles for policy development. Stakeholders believe that healthcare institutions rather than state or certifying boards should lead implementation of policies related to LCPs, yet expressed concerns about resistance by physicians and the ability of institutions to address politically complex medical staff challenges. Respondents recommended a coaching and professional development framework, with environmental changes, to maximise safety and career longevity of physicians as they age.

CONCLUSIONS

Key stakeholders express a desire for wider adoption of LCP standards, but foresee significant culture change and practical challenges ahead. Participants recommended that institutions lead this work, with support from regulatory stakeholders that endorse standards and create frameworks for policy adoption.

The Function of Disclosing Medical Errors: New Cultural Challenges for Physicians

A general consensus has been reached in health care organizations that the disclosure of medical errors can be a very powerful way to improve patients and physicians well-being and serves as a core component to high quality health care. This practice strongly encourages transparent communication with patients after medical errors or unanticipated outcomes. However, many countries, such as Brazil, do not have a culture of disclosing harmful errors to patients or standards emphasizing the importance of disclosing, taking responsibility, apologizing, and discussing the prevention of recurrences. Medical error is not discussed or approached during medical school. The stigma of error has a strong connection with value judgments, and emotional support for physicians does not exist. This paper suggests that open communication with the patient is essential. Guidance about error disclosure from health care organizations would be helpful for quality and patient safety and for health care professionals in countries like Brazil.

Implementing communication and resolution programs: Lessons learned from the first 200 hospitals

The Communication and Optimal Resolution toolkit contains implementation guidance for communication and resolution programs that are comprehensive, principled, and systematic approaches to the prevention and response to patient harm that includes the open and honest communication of inappropriate care to patients and families that is coupled with financial compensation and the commitment to future prevention. The toolkit was released by the Agency for Healthcare Research and Quality in May 2016. The authors describe their experiences with implementing communication and resolution programs with all of the components of Communication and Optimal Resolution and other communication and resolution program best practices in over 200 hospitals. Lessons learned include the ability to predict and discover actual barriers to full implementation and strategies for overcoming them are shared. Future considerations for further dissemination and spread are discussed.

The Aging Physician and the Medical Profession: A Review

Importance

The issue of the aging physician and when to cease practice has been controversial for many years. There are reports of prominent physicians who practiced after becoming dangerous in old age, but the profession has not demonstrated the ability to prevent this. A mandatory retirement age could be discriminatory and take many competent physicians out of practice and risk a physician shortage. An increasing body of evidence regarding the relationship between physicians' age and performance has led organizations, such as the American College of Surgeons, to revisit this challenge.

Observations

Since 1975, the number of practicing physicians older than 65 years in the United States has increased by more than 374%, and in 2015, 23% of practicing physicians were 65 years or older. Research shows that between ages 40 and 75 years, the mean cognitive ability declines by more than 20%, but there is significant variability from one person to another, indicating that while some older physicians are profoundly impaired, others retain their ability and skills. There are age-based requirements for periodic testing and/or retirement for many professions including pilots, judges, air traffic controllers, Federal Bureau of Investigation employees, and firefighters. While there are not similar requirements for physicians, a few hospitals have introduced mandatory age-based evaluations.

Conclusions

As physicians age, a required cognitive evaluation combined with a confidential, anonymous feedback evaluation by peers and coworkers regarding wellness and competence would be beneficial both to physicians and their patients. While it is unlikely that this will become a national standard soon, individual health care organizations could develop policies similar to those present at a few US institutions. In addition, large professional organizations should identify a range of acceptable policies to address the aging physician while leaving institutions flexibility to customize the approach. Absent robust professional initiatives in this area, regulators and legislators may impose more draconian measures.

Surgeons' Disclosures of Clinical Adverse Events

Importance

Surgeons are frequently faced with clinical adverse events owing to the nature of their specialty, yet not all surgeons disclose these events to patients. To sustain open disclosure programs, it is essential to understand how surgeons are disclosing adverse events, factors that are associated with reporting such events, and the effect of disclosure on surgeons.

Objective

To quantitatively assess surgeons' reports of disclosure of adverse events and aspects of their experiences with the disclosure process.

Design, Setting, and Participants

An observational study was conducted from January 1, 2011, to December 31, 2013, involving a 21-item baseline questionnaire administered to 67 of 75 surgeons (89%) representing 12 specialties at 3 Veterans Affairs medical centers. Sixty-two surveys of their communication about adverse events and experiences with disclosing such events were completed by 35 of these 67 surgeons (52%). Data were analyzed using mixed linear random-effects and logistic regression models.

Main Outcomes and Measures

Self-reports of disclosure assessed by 8 items from guidelines and pilot research, surgeons' perceptions of the adverse event, reported personal effects from disclosure, and baseline attitudes toward disclosure.

Results

Most of the surgeons completing the web-based surveys (41 responses from men and 21 responses from women) used 5 of the 8 recommended disclosure items: explained why the event happened (55 of 60 surveys [92%]), expressed regret for what happened (52 of 60 [87%]), expressed concern for the patient's welfare (57 of 60 [95%]), disclosed the adverse event within 24 hours (58 of 60 [97%]), and discussed steps taken to treat any subsequent problems (59 of 60 [98%]). Fewer surgeons apologized to patients (33 of 60 [55%]), discussed whether the event was preventable (33 of 60 [55%]), or how recurrences could be prevented (19 of 59 [32%]). Surgeons who were less likely to have discussed prevention (33 of 60 [55%]), those who stated the event was very or extremely serious (40 of 61 surveys [66%]), or reported very or somewhat difficult experiences discussing the event (16 of 61 [26%]) were more likely to have been negatively affected by the event. Surgeons with more negative attitudes about disclosure at baseline reported more anxiety about patients' surgical outcomes or events following disclosure (odds ratio, 1.54; 95% CI, 1.16-2.06).

Conclusions and Relevance

Surgeons who reported they were less likely to discuss preventability of the adverse event, or who reported difficult communication experiences, were more negatively affected by disclosure than others. Quality improvement efforts focused on recognizing the association between disclosure and surgeons' well-being may help sustain open disclosure policies.

Pathologists' Perspectives on Disclosing Harmful Pathology Error

CONTEXT

- Medical errors are unfortunately common. The US Institute of Medicine proposed guidelines for mitigating and disclosing errors. Implementing these recommendations in pathology will require a better understanding of how errors occur in pathology, the relationship between pathologists and treating clinicians in reducing error, and pathologists' experiences with and attitudes toward disclosure of medical error.

OBJECTIVE

- To understand pathologists' attitudes toward disclosing pathology error to treating clinicians and patients.

DESIGN

- We conducted 5 structured focus groups in Washington State and Missouri with 45 pathologists in academic and community practice. Participants were questioned about pathology errors, how clinicians respond to pathology errors, and what roles pathologists should play in error disclosure to patients.

RESULTS

- These pathologists believe that neither treating physicians nor patients understand the subtleties and limitations of pathologic diagnoses, which complicates discussions about pathology errors. Pathologists' lack of confidence in communication skills and fear of being misrepresented or misunderstood are major barriers to their participation in disclosure discussions. Pathologists see potential for their future involvement in disclosing error to patients, but at present advocate reliance on treating clinicians to disclose pathology errors to patients. Most group members believed that going forward pathologists should offer to participate more actively in error disclosure to patients.

CONCLUSIONS

- Pathologists lack confidence in error disclosure communication skills with both treating physicians and patients. Improved communication between pathologists and treating physicians could enhance transparency and promote disclosure of pathology errors. Consensus guidelines for best practices in pathology error disclosure may be useful.

Implementing an error disclosure coaching model: A multicenter case study

National guidelines call for health care organizations to provide around-the-clock coaching for medical error disclosure. However, frontline clinicians may not always seek risk managers for coaching. As part of a demonstration project designed to improve patient safety and reduce malpractice liability, we trained multidisciplinary disclosure coaches at 8 health care organizations in Washington State. The training was highly rated by participants, although not all emerged confident in their coaching skill. This multisite intervention can serve as a model for other organizations looking to enhance existing disclosure capabilities. Success likely requires cultural change and repeated practice opportunities for coaches.

Risk managers' descriptions of programs to support second victims after adverse events

Guidelines call for healthcare organizations to provide emotional support for clinicians involved in adverse events, but little is known about how these organizations seek to meet this need. We surveyed US members of the American Society for Healthcare Risk Management (ASHRM) about the presence, features, and perceived efficacy of their organization's provider support program. The majority reported that their organization had a support program, but features varied widely and there are substantial opportunities to improve services. Provider support programs should enhance referral mechanisms and peer support, critically appraise the role of employee assistance programs, and demonstrate their value to institutional leaders.

Evaluating the implementation of a national disclosure policy for large-scale adverse events in an integrated health care system: identification of gaps and successes

Background

Many healthcare organizations have developed disclosure policies for large-scale adverse events, including the Veterans Health Administration (VA). This study evaluated VA’s national large-scale disclosure policy and identifies gaps and successes in its implementation.

Methods

Semi-structured qualitative interviews were conducted with leaders, hospital employees, and patients at nine sites to elicit their perceptions of recent large-scale adverse events notifications and the national disclosure policy. Data were coded using the constructs of the Consolidated Framework for Implementation Research (CFIR).

Results

We conducted 97 interviews. Insights included how to handle the communication of large-scale disclosures through multiple levels of a large healthcare organization and manage ongoing communications about the event with employees. Of the 5 CFIR constructs and 26 sub-constructs assessed, seven were prominent in interviews. Leaders and employees specifically mentioned key problem areas involving 1) networks and communications during disclosure, 2) organizational culture, 3) engagement of external change agents during disclosure, and 4) a need for reflecting on and evaluating the policy implementation and disclosure itself. Patients shared 5) preferences for personal outreach by phone in place of the current use of certified letters. All interviewees discussed 6) issues with execution and 7) costs of the disclosure.

Conclusions

CFIR analysis reveals key problem areas that need to be addresses during disclosure, including: timely communication patterns throughout the organization, establishing a supportive culture prior to implementation, using patient-approved, effective communications strategies during disclosures; providing follow-up support for employees and patients, and sharing lessons learned.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-016-1903-7) contains supplementary material, which is available to authorized users.

Challenges of Implementing a Communication-and-Resolution Program Where Multiple Organizations Must Cooperate

OBJECTIVE

To implement a communication-and-resolution program (CRP) in a setting in which liability insurers and health care facilities must collaborate to resolve incidents involving a facility and separately insured clinicians.

STUDY SETTING

Six hospitals and clinics and a liability insurer in Washington State.

STUDY DESIGN

Sites designed and implemented CRPs and contributed information about cases and operational challenges over 20 months. Data were qualitatively analyzed.

DATA COLLECTION METHODS

Data from interviews with personnel responsible for CRP implementation were triangulated with data on program cases collected by sites and notes recorded during meetings with sites and among project team members.

PRINCIPAL FINDINGS

Sites experienced small victories in resolving particular cases and streamlining some working relationships, but they were unable to successfully implement a collaborative CRP. Barriers included the insurer's distance from the point of care, passive rather than active support from top leaders, coordinating across departments and organizations, workload, nonparticipation by some physicians, and overcoming distrust.

CONCLUSIONS

Operating CRPs where multiple organizations must collaborate can be highly challenging. Success likely requires several preconditions, including preexisting trust among organizations, active leadership engagement, physicians' commitment to participate, mechanisms for quickly transmitting information to insurers, tolerance for missteps, and clear protocols for joint investigations and resolutions.

Improving Communication and Resolution Following Adverse Events Using a Patient-Created Simulation Exercise

OBJECTIVE

The response to adverse events can lack patient-centeredness, perhaps because the involved institutions and other stakeholders misunderstand what patients and families go through after care breakdowns.

STUDY SETTING

Washington and Texas.

STUDY DESIGN

The HealthPact Patient and Family Advisory Council (PFAC) created and led a five-stage simulation exercise to help stakeholders understand what patients experience following an adverse event. The half-day exercise was presented twice.

DATA COLLECTION AND ANALYSIS

Lessons learned related to the development and conduct of the exercise were synthesized from planning notes, attendee evaluations, and exercise discussion notes.

PRINCIPAL FINDINGS

One hundred ninety-four individuals attended (86 Washington and 108 Texas). Take-homes from these exercises included the fact that the response to adverse events can be complex, siloed, and uncoordinated. Participating in this simulation exercise led stakeholders and patient advocates to express interest in continued collaboration.

CONCLUSIONS

A PFAC-designed simulation can help stakeholders understand patient and family experiences following adverse events and potentially improve their response to these events.

Patients as Partners in Learning from Unexpected Events

IMPORTANCE

Patient safety experts believe that patients/family members should be involved in adverse event review. However, it is unclear how aware patients/family members are about the causes of adverse events they experienced.

OBJECTIVE

To determine whether patients/family members interviewed could identify at least one contributing factor for the event they experienced. Secondary objectives included understanding the way patients/family members became aware of adverse events, the types of contributing factors patients/family members identified for different types of adverse events, and recommendations provided by patients/family members to address the contributing factors.

DESIGN

We interviewed patients/family members using semistructured interviews to understand their perceptions about why these adverse events occurred. The adverse events occurred between 1991 and 2014.

SETTING

Participants described adverse events that occurred in various types of health care organizations (i.e., hospitals, ambulatory facilities/clinics, and dental clinics).

PARTICIPANTS

We interviewed 72 patients and family members who each described a unique adverse event. Eligibility requirements were that patients/family members spoke English or Spanish and were aware of an adverse event that happened to them or a loved one. INTERVENTION(S) FOR CLINICAL TRIALS OR EXPOSURE(S) FOR OBSERVATIONAL STUDIES: N/A.

MAIN OUTCOME(S) AND MEASURE(S)

The main outcome was determining whether patients/family members could identify at least one contributing factor they perceived as related to the adverse event they described.

RESULTS

Each participant identified at least one contributing factor and on average identified 3.67 contributing factors for their event. The most frequently mentioned contributing factors were Staff Qualifications/Knowledge (79 percent), Safety Policies/Procedures (74 percent), and Communication (64 percent). Participants knew about the contributing factors from personal observation only (32 percent), personal reasoning (11 percent), personal research (7 percent), record review (either their own medical records or reports they received in their own investigation; 6 percent), and being told by a physician (5 percent). Finally, patients/family members were able to provide recommendations that address each of the nine contributing factors we examined.

CONCLUSIONS AND RELEVANCE

Patients/family members identified contributing factors related to their adverse event. Given that these contributing factors might not be known to health care organizations because most participants stated that they were not involved in the analysis process, opportunities for organizational learning from patients are potentially being missed. Health care organizations should interview patients/family about the event that harmed them to help ensure a full understanding of the causes of the event.

Collaboration with Regulators to Support Quality and Accountability Following Medical Errors: The Communication and Resolution Program Certification Pilot

OBJECTIVE

Communication and resolution programs (CRPs) involve institutions responding to adverse events using transparency with patients, event analysis, recurrence prevention, and compensation. Collaboration with regulators around CRPs could enhance health care quality.

SETTING AND PARTICIPANTS

Health care institutions, liability insurers, and the Medical Quality Assurance Commission (MQAC, board of medicine) in Washington State.

STUDY DESIGN

MQAC has collaborated with the Foundation for Health Care Quality (FHCQ) on the CRP Certification pilot. A panel of physicians, risk managers, and patient advocates at FHCQ will review cases for use of the CRP key elements. Cases meeting this standard will be "CRP Certified." If MQAC determines that the CRP enhanced patient safety comparable or better than board action, the Commission may close the case.

PRINCIPAL FINDINGS

Developing this process identified the following issues: (1) protecting information submitted for CRP Certification; (2) determining what information the Commission needs to assess whether additional investigation is warranted; (3) preserving the Commission's responsibility to protect the public while working with health care organizations; and (4) addressing concerns that CRP Certification not shield incompetent providers.

CONCLUSIONS

The CRP Certification program is a promising example of collaboration among institutions, insurers, and regulators to promote patient-centered accountability and learning following adverse events.

The experiences of risk managers in providing emotional support for health care workers after adverse events

Risk managers often meet with health care workers who are emotionally traumatized following adverse events. We surveyed members of the American Society for Health care Risk Management (ASHRM) about their training, experience, competence, and comfort with providing emotional support to health care workers. Although risk managers reported feeling comfortable and competent in providing support, nearly all respondents prefer to receive additional training. Risk managers who were comfortable listening to and supporting health care workers were more likely to report prior training. Health care organizations implementing second victim support programs should not rely solely on risk managers to provide support, rather engage and train interested risk managers and provide them with opportunities to practice.

Disclosure of adverse events and errors in surgical care: challenges and strategies for improvement

The disclosure of adverse events to patients, including those caused by medical errors, is a critical part of patient-centered healthcare and a fundamental component of patient safety and quality improvement. Disclosure benefits patients, providers, and healthcare institutions. However, the act of disclosure can be difficult for physicians. Surgeons struggle with disclosure in unique ways compared with other specialties, and disclosure in the surgical setting has specific challenges. The frequency of surgical adverse events along with a dysfunctional tort system, the team structure of surgical staff, and obstacles created inadvertently by existing surgical patient safety initiatives may contribute to an environment not conducive to disclosure. Fortunately, there are multiple strategies to address these barriers. Participation in communication and resolution programs, integration of Just Culture principles, surgical team disclosure planning, refinement of informed consent and morbidity and mortality processes, surgery-specific professional standards, and understanding the complexities of disclosing other clinicians' errors all have the potential to help surgeons provide patients with complete, satisfactory disclosures. Improvement in the regularity and quality of disclosures after surgical adverse events and errors will be key as the field of patient safety continues to advance.

Attitudes Toward Risk and Informed Consent for Research on Medical Practices: A Cross-sectional Survey

BACKGROUND

The U.S. Office for Human Research Protections has proposed that end points of randomized trials comparing the effectiveness of standard medical practices are risks of research that would require disclosure and written informed consent, but data are lacking on the views of potential participants.

OBJECTIVE

To assess attitudes of U.S. adults about risks and preferences for notification and consent for research on medical practices.

DESIGN

Cross-sectional survey conducted in August 2014.

SETTING

Web-based questionnaire.

PATIENTS

1095 U.S. adults sampled from an online panel (n = 805) and an online convenience river sample (n = 290).

MEASUREMENTS

Attitudes toward risk, informed consent, and willingness to participate in 3 research scenarios involving medical record review and randomization of usual medical practices.

RESULTS

97% of respondents agreed that health systems should evaluate standard treatments. Most wanted to be asked for permission to participate in each of 3 scenarios (range, 75.2% to 80.4%), even if it involved only medical record review, but most would accept nonwritten (oral) permission or general notification if obtaining written permission would make the research too difficult to conduct (range, 70.2% to 82.7%). Most perceived additional risk from each scenario (range, 64.0% to 81.6%).

LIMITATION

Use of hypothetical scenarios and a nonprobability sample that was not fully representative of the U.S. population.

CONCLUSION

Most respondents preferred to be asked for permission to participate in observational and randomized research evaluating usual medical practices, but they are willing to accept less elaborate approaches than written consent if research would otherwise be impracticable. These attitudes are not aligned with proposed regulatory guidance.

PRIMARY FUNDING SOURCE

National Center for Advancing Translational Sciences at the National Institutes of Health.