Implementation of an Online Reporting System to Identify Unprofessional Behaviors and Mistreatment Directed at Trainees at an Academic Medical Center

IMPORTANCE

Unprofessional behaviors and mistreatment directed at trainees continue to challenge the learning environment. Academic medical institutions should encourage reports of inappropriate behavior and address such reports directly to create a safe learning environment.

OBJECTIVE

To determine the feasibility of creating and implementing an online reporting system for receiving and reviewing complaints of unprofessional behavior directed toward or experienced by students, postdoctoral trainees, and residents.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study assessed implementation of an online reporting system (feedback form) with a method for triaging reports, providing both positive and negative feedback, as well as adjudication and transparent public disclosure of aggregate data. The system was launched at a large urban academic medical center with numerous trainees that is fully integrated with a health system of 8 hospitals. Participants included faculty who interact with trainees, medical students, graduate students and postdoctoral fellows, and residents and clinical fellows. Follow-up began in October 2019 (at the time of tool launch) and lasted through December 2021. Data were analyzed from January to March 2022.

MAIN OUTCOMES AND MEASURES

The primary outcomes were the numbers and types of reports according to the reporter and the person reported about.

RESULTS

Participants included 2900 faculty who interact with trainees, 600 medical students, more than 1000 graduate students and postdoctoral fellows, and 2600 residents and clinical fellows. Trainees submitted 196 reports, 173 (88.3%) of which described unprofessional interactions. Among the reports describing unprofessional behavior, 60 (34.7%) were from medical students, 96 (55.5%) were from residents and fellows, 17 (9.8%) were from graduate students or postdoctoral trainees, and 78 (45.1%) were from men. The majority of negative reports described behaviors by faculty (106 [61.3%]), followed by residents and fellows (24 [13.9%]). Twenty faculty (<1.0%) accounted for 52 (50.0%) of the 104 reports describing unprofessional behaviors. Since implementation, most trainees are aware of this process. An increasing number have reported instances of mistreatment, and those who shared concerns through the online system report satisfaction with the outcome of the response to the report.

CONCLUSIONS AND RELEVANCE

In this cohort study, the new reporting mechanism facilitated identification of the small number of individuals associated with unprofessional behaviors toward trainees and increased awareness of the school's commitment to creating a safe learning environment.

Qualitative Content Analysis of Coworkers' Safety Reports of Unprofessional Behavior by Physicians and Advanced Practice Professionals

OBJECTIVES

The aims of the study were to develop a valid and reliable taxonomy of coworker reports of alleged unprofessional behavior by physicians and advanced practice professionals and determine the prevalence of reports describing particular types of unprofessional conduct.

METHODS

We conducted qualitative content analysis of coworker reports of alleged unprofessional behavior by physicians and advanced practice professionals to create a standardized taxonomy. We conducted a focus group of experts in medical professionalism to assess the taxonomy's face validity. We randomly selected 120 reports (20%) of the 590 total reports submitted through the medical center's safety event reporting system between June 2015 and September 2016 to measure interrater reliability of taxonomy codes and estimate the prevalence of reports describing particular types of conduct.

RESULTS

The initial taxonomy contained 22 codes organized into the following four domains: competent medical care, clear and respectful communication, integrity, and responsibility. All 10 experts agreed that the four domains reflected essential elements of medical professionalism. Interrater reliabilities for all codes and domains had a κ value greater than the 0.60 threshold for good reliability. Most reports (60%, 95% confidence interval = 51%-69%) described disrespectful or offensive communication. Nine codes had a prevalence of less than 1% and were folded into their respective domains resulting in a final taxonomy composed of 13 codes.

CONCLUSIONS

The final taxonomy represents a useful tool with demonstrated validity and reliability, opening the door for reliable analysis and systems to promote accountability and behavior change. Given the safety implications of unprofessional behavior, understanding the typology of coworker observations of unprofessional behavior may inform organization strategies to address this threat to patient safety.

Association of Coworker Reports About Unprofessional Behavior by Surgeons With Surgical Complications in Their Patients

Importance

For surgical teams, high reliability and optimal performance depend on effective communication, mutual respect, and continuous situational awareness. Surgeons who model unprofessional behaviors may undermine a culture of safety, threaten teamwork, and thereby increase the risk for medical errors and surgical complications.

Objective

To test the hypothesis that patients of surgeons with higher numbers of reports from coworkers about unprofessional behaviors are at greater risk for postoperative complications than patients whose surgeons generate fewer coworker reports.

Design, Setting, and Participants

This retrospective cohort study assessed data from 2 geographically diverse academic medical centers that participated in the National Surgical Quality Improvement Program (NSQIP) and recorded and acted on electronic reports of safety events from coworkers describing unprofessional behavior by surgeons. Patients included in the NSQIP database who underwent inpatient or outpatient operations at 1 of the 2 participating sites from January 1, 2012, through December 31, 2016, were eligible. Patients were excluded if they were younger than 18 years on the date of the operation or if the attending surgeon had less than 36 months of monitoring for coworker reports preceding the date of the operation. Data were analyzed from August 8, 2018, through April 9, 2019.

Exposures

Coworker reports about unprofessional behavior by the surgeon in the 36 months preceding the date of the operation.

Main Outcomes and Measures

Postoperative surgical or medical complications, as defined by the NSQIP, within 30 days of the operation.

Results

Among 13 653 patients in the cohort (54.0% [7368 ] female; mean [SD] age, 57 [16] years) who underwent operations performed by 202 surgeons (70.8% [143] male), 1583 (11.6%) experienced a complication, including 825 surgical (6.0%) and 1070 medical (7.8%) complications. Patients whose surgeons had more coworker reports were significantly more likely to experience any complication (0 reports, 954 of 8916 [10.7%]; ≥4 reports, 294 of 2087 [14.1%]; P < .001), any surgical complication (0 reports, 516 of 8916 [5.8%]; ≥4 reports, 159 of 2087 [7.6%]; P < .01), or any medical complication (0 reports, 634 of 8916 [7.1%]; ≥4 reports, 196 of 2087 [9.4%]; P < .001). The adjusted complication rate was 14.3% higher for patients whose surgeons had 1 to 3 reports and 11.9% higher for patients whose surgeons had 4 or more reports compared with patients whose surgeons had no coworker reports (P = .05).

Conclusions and Relevance

Patients whose surgeons had higher numbers of coworker reports about unprofessional behavior in the 36 months before the patient's operation appeared to be at increased risk of surgical and medical complications. These findings suggest that organizations interested in ensuring optimal patient outcomes should focus on addressing surgeons whose behavior toward other medical professionals may increase patients' risk for adverse outcomes.

Unsolicited Patient Complaints Identify Physicians with Evidence of Neurocognitive Disorders

OBJECTIVES

Determine whether words contained in unsolicited patient complaints differentiate physicians with and without neurocognitive disorders (NCD).

METHODS

We conducted a nested case-control study using data from 144 healthcare organizations that participate in the Patient Advocacy Reporting System program. Cases (physicians with probable or possible NCD) and two comparison groups of 60 physicians each (matched for age/sex and site/number of unsolicited patient complaints) were identified from 33,814 physicians practicing at study sites. We compared the frequency of words in patient complaints related to an NCD diagnostic domain between cases and our two comparison groups.

RESULTS

Individual words were all statistically more likely to appear in patient complaints for cases (73% of cases had at least one such word) compared to age/sex matched (8%, p < 0.001 using Pearson's χ² test, χ² = 30.21, df = 1) and site/complaint matched comparisons (18%, p < 0.001 using Pearson's χ² test, χ² = 17.51, df = 1). Cases were significantly more likely to have at least one complaint with any word describing NCD than the two comparison groups combined (conditional logistic model adjusted odds ratio 20.0 [95% confidence interval 4.9-81.7]).

CONCLUSIONS

Analysis of words in unsolicited patient complaints found that descriptions of interactions with physicians with NCD were significantly more likely to include words from one of the diagnostic domains for NCD than were two different comparison groups. Further research is needed to understand whether patients might provide information for healthcare organizations interested in identifying professionals with evidence of cognitive impairment.

Use of Unsolicited Patient Observations to Identify Surgeons With Increased Risk for Postoperative Complications

Importance

Unsolicited patient observations are associated with risk of medical malpractice claims. Because lawsuits may be triggered by an unexpected adverse outcome superimposed on a strained patient-physician relationship, a question remains as to whether behaviors that generate patient dissatisfaction might also contribute to the genesis of adverse outcomes themselves.

Objective

To examine whether patients of surgeons with a history of higher numbers of unsolicited patient observations are at greater risk for postoperative complications than patients whose surgeons generate fewer such unsolicited patient observations.

Design, Setting, and Participants

This retrospective cohort study used data from 7 academic medical centers participating in the National Surgical Quality Improvement Program and the Vanderbilt Patient Advocacy Reporting System from January 1, 2011, to December 31, 2013. Patients older than 18 years included in the National Surgical Quality Improvement Program who underwent inpatient or outpatient operations at 1 of the participating sites during the study period were included. Patients were excluded if the attending surgeon had less than 24 months of data in the Vanderbilt Patient Advocacy Reporting System preceding the date of the operation. Data analysis was conducted from June 1, 2015, to October 20, 2016.

Exposures

Unsolicited patient observations for the patient's surgeon in the 24 months preceding the date of the operation.

Main Outcomes and Measures

Postoperative surgical or medical complications as defined by the National Surgical Quality Improvement Program within 30 days of the operation of interest.

Results

Among the 32 125 patients in the cohort (13 230 men, 18 895 women; mean [SD] age, 55.8 [15.8] years), 3501 (10.9%) experienced a complication, including 1754 (5.5%) surgical and 2422 (7.5%) medical complications. Prior unsolicited patient observations for a surgeon were significantly associated with the risk of a patient having any complication (odds ratio, 1.0063; 95% CI, 1.0004-1.0123; P = .03), any surgical complication (odds ratio, 1.0104; 95% CI, 1.0022-1.0186; P = .01), any medical complication (odds ratio, 1.0079; 95% CI, 1.0009-1.0148; P = .03), and being readmitted (odds ratio, 1.0088, 95% CI, 1.0024-1.0151; P = .007). The adjusted rate of complications was 13.9% higher for patients whose surgeon was in the highest quartile of unsolicited patient observations compared with patients whose surgeon was in the lowest quartile.

Conclusions and Relevance

Patients whose surgeons have large numbers of unsolicited patient observations in the 24 months prior to the patient's operation are at increased risk of surgical and medical complications. Efforts to promote patient safety and address risk of malpractice claims should continue to focus on surgeons' ability to communicate respectfully and effectively with patients and other medical professionals.

Triaging Patient Complaints: Monte Carlo Cross-Validation of Six Machine Learning Classifiers

Background

Unsolicited patient complaints can be a useful service recovery tool for health care organizations. Some patient complaints contain information that may necessitate further action on the part of the health care organization and/or the health care professional. Current approaches depend on the manual processing of patient complaints, which can be costly, slow, and challenging in terms of scalability.

Objective

The aim of this study was to evaluate automatic patient triage, which can potentially improve response time and provide much-needed scale, thereby enhancing opportunities to encourage physicians to self-regulate.

Methods

We implemented a comparison of several well-known machine learning classifiers to detect whether a complaint was associated with a physician or his/her medical practice. We compared these classifiers using a real-life dataset containing 14,335 patient complaints associated with 768 physicians that was extracted from patient complaints collected by the Patient Advocacy Reporting System developed at Vanderbilt University and associated institutions. We conducted a 10-splits Monte Carlo cross-validation to validate our results.

Results

We achieved an accuracy of 82% and F-score of 81% in correctly classifying patient complaints with sensitivity and specificity of 0.76 and 0.87, respectively.

Conclusions

We demonstrate that natural language processing methods based on modeling patient complaint text can be effective in identifying those patient complaints requiring physician action.

Using Coworker Observations to Promote Accountability for Disrespectful and Unsafe Behaviors by Physicians and Advanced Practice Professionals

BACKGROUND

Health care team members are well positioned to observe disrespectful and unsafe conduct-behaviors known to undermine team function. Based on experience in sharing patient complaints with physicians who subsequently achieved decreased complaints and malpractice risk, Vanderbilt University Medical Center developed and assessed the feasibility of the Co-Worker Observation Reporting System(SM) (CORS (SM)) for addressing coworkers' reported concerns.

METHODS

VUMC leaders used a "Project Bundle" readiness assessment, which entailed identification and development of key people, organizational supports, and systems. Methods involved gaining leadership buy-in, recruiting and training key individuals, aligning the project with organizational values and policies, promoting reporting, monitoring reports, and employing a tiered intervention process to address reported coworker concerns.

RESULTS

Peer messengers shared coworker reports with the physicians and advanced practice professionals associated with at least one report 84% of the time. Since CORS inception, 3% of the medical staff was associated with a pattern of CORS reports, and 71% of recipients of pattern-related interventions were not named in any subsequent reports in a one-year follow-up period.

CONCLUSIONS

Systematic monitoring of documented co-worker observations about unprofessional conduct and sharing that information with involved professionals are feasible. Feasibility requires organizationwide implementation; co-workers willing and able to share respectful, nonjudgmental, timely feedback designed initially to encourage self-reflection; and leadership committed to be more directive if needed. Follow-up surveillance indicates that the majority of professionals "self-regulate" after receiving CORS data.

An intervention model that promotes accountability: peer messengers and patient/family complaints

BACKGROUND

Patients and their families are well positioned to partner with health care organizations to help identify unsafe and dissatisfying behaviors and performance. A peer messenger process was designed by the Center for Professional and Patient Advocacy at Vanderbilt University Medical Center (Nashville, Tennessee) to address "high-risk" physicians identified through analysis of unsolicited patient complaints, a proxy for risk of lawsuits.

METHODS

This retrospective, descriptive study used peer messenger debriefing results from data-driven interventions at 16 geographically disparate community (n = 7) and academic (n = 9) medical centers in the United States. Some 178 physicians served as peer messengers, conducting interventions from 2005, through 2009 on 373 physicians identified as high risk.

RESULTS

Most (97%) of the high-risk physicians received the feedback professionally, and 64% were "Responders." Responders' risk scores improved at least 15%, where Nonresponders' scores worsened (17%) or remained unchanged (19%) (p < or = .001). Responders were more often physicians practicing in medicine and surgery than emergency medicine physicians, had longer organizational tenures, and engaged in lengthier first-time intervention meetings with messengers. Years to achieve responder status correlated positively with initial communication-related complaints (r = .32, p < .001), but all complaint categories were equally likely to change over time.

CONCLUSIONS

Peer messengers, recognized by leaders and appropriately supported with ongoing training, high-quality data, and evidence of positive outcomes, are willing to intervene with colleagues over an extended period of time. The physician peer messenger process reduces patient complaints and is adaptable to addressing unnecessary variation in other quality/safety metrics.

A strategy for determining arterial blood gases on the summit of Mt. Everest

Background

Climbers on the summit of Mt. Everest are exposed to extreme hypoxia, and the physiological implications are of great interest. Inferences have been made from alveolar gas samples collected on the summit, but arterial blood samples would give critical information. We propose a plan to insert an arterial catheter at an altitude of 8000 m, take blood samples above this using an automatic sampler, store the samples in glass syringes in an ice-water slurry, and analyze them lower on the mountain 4 to 6 hours later.

Results

A preliminary design of the automatic sampler was successfully tested at the White Mountain Research Station (altitude 3800 m – 4300 m). To determine how much the blood gases changed over a long period, rabbit blood was tonometered to give a gas composition close to that expected on the summit (PO₂ 4.0 kPa (30 mmHg), PCO₂ 1.3 kPa (10 mmHg), pH 7.7) and the blood gases were measured every 2 hours for 8 hours both at sea level and 3800 m. The mean changes were PO₂ +0.3 to +0.4 kPa (+2 to +3 mmHg), PCO₂ 0 to +0.13 kPa (+1 mmHg), pH -0.02 to -0.04, base excess -0.7 to -1.2 mM. In practice the delay before analysis should not exceed 4 to 6 hours. The small paradoxical rise in PO₂ is presumably caused mainly by contamination of the blood with air.

Conclusion

We conclude that automatic arterial blood sampling at high altitude is technically feasible and that the changes in the blood gases over a period of several hours are acceptably small.

How I feel: a self-report measure of emotional arousal and regulation for children

This article details the development and preliminary validation of a multidimensional self-report measure of emotion for 8- to 12-year-old children--the How I Feel (HIF). Item generation and selection occurred via 2 pilot administrations (ns = 250 and 378, respectively). Ten experts provided data on content validity. Exploratory factor analysis and subsequent confirmatory factor analysis with samples of 406, 524, 349, and 349 3rd-through 6th-grade children supported a 3-factor model, including the frequency and intensity of (a) positive emotion, (b) negative emotion, and (c) positive and negative emotion control. Results showed moderate longitudinal stability for 120 children over 2 years. Concurrent validity was established. The HIF can be useful in understanding the interplay between arousal and control in social-emotional adjustment in school-age children.

Characterization of cardiotoxicity induced by 2,3,7, 8-tetrachlorodibenzo-p-dioxin and related chemicals during early chick embryo development

Cardiotoxicity of 2,3,7,8-tetrachlorodibenzo-p-dioxin (TCDD) was studied in White Leghorn-Babcock (WLB) and Plymouth Rock-Barred (PRB) chick embryos. TCDD, injected on day 0 (D0), induced a dose-related increase in heart weight in both strains in the absence of pericardial edema on D10. PRB embryos were four to five times more sensitive to this cardiotoxicity than WLB. To determine if another aryl hydrocarbon receptor agonist produced a similar response, graded doses of TCDD; 3,3',4,4',5-pentachlorobiphenyl (PeCB 126); or 2,2',4,4',5,5'-hexachlorobiphenyl (HxCB 153) were injected into WLB eggs. TCDD and PeCB 126 induced a dose-related increase in heart weight without pericardial edema, while HxCB 153 had no effect. We then hypothesized that TCDD-induced cardiotoxicity progressed to heart failure and edema. In PRB, morphometric analysis revealed that TCDD (0.06-0.45 pmol/g) induced a dose-related increase in left and right ventricle cavity area without wall hypertrophy on D10, consistent with dilated cardiomyopathy. A time course showed that 0.24 pmol/g did not alter heart morphology on D8 but induced cardiac dilation on D10 and D12. The 0.24 pmol/g dose also induced changes associated with progression of cardiomyopathy toward heart failure, including increased cardiac atrial natriuretic factor mRNA expression and decreased cardiac responsiveness to isoproterenol-induced positive chronotropy, on D10 and D12. Finally, 0.24 pmol/g induced a significantly higher incidence of subcutaneous and peritoneal edema, indicative of overt heart failure, on D12 (75%, 15/20) compared to D10 (14%, 3/22). In conclusion, TCDD induced a phenotype of dilated cardiomyopathy and symptoms associated with the development of congestive heart failure.