A review of current approaches for evaluating impaired performance in around-the-clock medical professionals

Doctors' alertness, contentedness and calmness before and after night shifts: a latent profile analysis

BACKGROUND

While night shifts are crucial for patient care, they threaten doctors' well-being and performance. Knowledge of how the impact of night shifts differs for doctors is needed to attenuate the adverse effects of night shifts. This study aimed to obtain more precise insight into doctors' feelings surrounding night shift by: identifying profiles based on doctors' alertness, contentedness and calmness scores before and after night shifts (research question (RQ) 1); assessing how doctors' pre- and post-shift profiles change (RQ2); and determining associations of doctors' demographics and shift circumstances with alertness, contentedness and calmness change (RQ3).

METHODS

Latent Profile Analysis using doctors' pre- and post-shift self-rated alertness, contentedness and calmness scores was employed to identify pre- and post-shift profiles (RQ1). A cross-tabulation revealed pre- and post-shift profile changes (RQ2). Multiple regressions determined associations of demographics (i.e. age, sex, specialty) and night shift circumstances (i.e. hours worked pre-call, hours awake pre-call, shift duration, number of consecutive shifts, total hours of sleep) with alertness, contentedness and calmness change (RQ3).

RESULTS

In total, 211 doctors participated with a mean age of 39.8 ± 10 years; 47.4% was male. The participants included consultants (46.4%) and trainees (53.6%) of the specialties surgery (64.5%) and obstetrics/gynaecology (35.5%). Three pre-shift (Indifferent, Ready, Engaged) and four post-shift profiles (Lethargic, Tired but satisfied, Excited, Mindful) were found. Most doctors changed from Ready to Tired but satisfied, with alertness reducing most. Age, specialty, sleep, shift duration and the number of consecutive shifts associated with alertness, contentedness and calmness changes.

CONCLUSIONS

The results provided nuanced insight into doctors' feelings before and after night shifts. Future research may assess whether specific subgroups benefit from tailored interventions.

Validation of RUBY for Breast Cancer Knowledge Extraction From a Large French Electronic Medical Record System

PURPOSE

RUBY is a tool for extracting clinical data on breast cancer from French medical records on the basis of named entity recognition models combined with keyword extraction and postprocessing rules. Although initial results showed a high precision of the system in extracting clinical information from surgery, pathology, and biopsy reports (≥92.7%) and good precision in extracting data from consultation reports (81.8%), its validation is needed before its use in routine practice.

METHODS

In this work, we analyzed RUBY's performance compared with the manual entry and we evaluated the generalizability of the approach on different sets of reports collected on a span of 40 years.

RESULTS

RUBY performed similarly or better than the manual entry for 15 of 27 variables. It showed similar performances when structuring newer reports but failed to extract entities for which changes in terminology appeared. Finally, our tool could automatically structure 15,990 reports in 77 minutes.

CONCLUSION

RUBY can automate the data entry process of a set of variables and reduce its burden, but a continuous evaluation of the format and structure of the reports and a subsequent update of the system is necessary to ensure its robustness.

Illusion of knowledge in statistics among clinicians: evaluating the alignment between objective accuracy and subjective confidence, an online survey

Healthcare professionals' statistical illiteracy can impair medical decision quality and compromise patient safety. Previous studies have documented clinicians' insufficient proficiency in statistics and a tendency in overconfidence. However, an underexplored aspect is clinicians' awareness of their lack of statistical knowledge that precludes any corrective intervention attempt. Here, we investigated physicians', residents' and medical students' alignment between subjective confidence judgments and objective accuracy in basic medical statistics. We also examined how gender, profile of experience and practice of research activity affect this alignment, and the influence of problem framing (conditional probabilities, CP vs. natural frequencies, NF). Eight hundred ninety-eight clinicians completed an online survey assessing skill and confidence on three topics: vaccine efficacy, p value and diagnostic test results interpretation. Results evidenced an overall consistent poor proficiency in statistics often combined with high confidence, even in incorrect answers. We also demonstrate that despite overconfidence bias, clinicians show a degree of metacognitive sensitivity, as their confidence judgments discriminate between their correct and incorrect answers. Finally, we confirm the positive impact of the more intuitive NF framing on accuracy. Together, our results pave the way for the development of teaching recommendations and pedagogical interventions such as promoting metacognition on basic knowledge and statistical reasoning as well as the use of NF to tackle statistical illiteracy in the medical context.

Assessment of fitness to perform using a validated self-test in obstetric and gynecological night shifts in the Netherlands

BACKGROUND

The field of obstetrics and gynecology requires complex decision-making and skills because of unexpected high-risk situations. These skills are influenced by alertness, reaction time, and concentration. Night shifts result in sleep deprivation, which might impair these functions, although it is still unclear to what extent.

OBJECTIVE

This study aimed to investigate whether a night shift routinely impairs the obstetrics and gynecology consultants' and residents' fitness to perform and whether this reaches a critical limit compared with relevant frames of reference.

STUDY DESIGN

Residents (n=33) and consultants (n=46) in obstetrics and gynecology conducted multiple measurements (n=415) at precall, postcall, and noncall moments with the fitness to perform self-test. The self-test consists of an adaptive pursuit tracking task that is able to objectively measure alertness, reaction time, concentration, and hand-eye coordination and Visual Analog Scale tests to subjectively score alertness. The test is validated with a sociolegal reference of a 0.06% ethanol blood concentration (the peak level after 2 units of alcohol, the legal driving limit). This equals -1.37% on the objective score and -8.17 points on subjective alertness. Linear mixed models were used to analyze the difference within subjects over a night shift, integrating repeated measures over time.

RESULTS

The overnight objective difference between postcall and precall measurements was -0.62 (P<.05) for residents and 0.28 (P=NS) for consultants, both not exceeding the sociolegal reference as a group. Objective impairment exceeded the reference for 31% of the residents and 28% of the consultants. Subjective alertness decreased in residents (-18.26; P<.001) and consultants (-10.85; P<.001), both exceeding the reference. No residents had to continue work postcall versus 7.8% of the consultants. None of the consultants that had to continue work were in an objective critically impaired state.

CONCLUSION

This study provides insight and awareness of individual performance after night shifts with clear frames of reference. The performance of residents is negatively and significantly affected by night shifts; therefore, a scheduled day off after a night shift is justified. Consultants showed no overall impairment; however, a quarter did exceed the alcohol limit reference after their night shift. If not logistically feasible to schedule a protected day off after a night shift, our group recommends safe shift scheduling, including options to transfer care after a demanding night shift to prevent working in a compromised state.

Night shift work in resident physicians: does it affect mood states and cognitive levels?

BACKGROUND

The effects of night shift work on health status have been widely studied. Night workers seem to smoke more, eat badly and show a low propensity to physical activity. Night work can be associated with an increase in cardiovascular and gastrointestinal disorders, alterations in immune response, diabetes, aging, hormonal imbalance, and premature death; alteration of circadian rhythm is also regarded as a risk factor for breast cancer and neuropsychiatric disorders. Moreover, several studies have highlighted the effects of sleep deprivation on clinical performance, quality of care and personal safety of healthcare personnel. No studies have investigated the effects of night work on Italian resident physicians and compared its effect across specialties. This study aims to assess the prevalence of sleep disorders, possible cognitive impairment and mood states, in relation to night shift work among resident physicians.

METHODS

80 resident physicians, attending the postgraduate training into an Hospital located in the South of Italy, were divided into 4 areas (medical, surgical, services and anaesthesia). They were recruited from July 2017 to June 2018 and participated to a survey consisting of 4 questionnaires to investigate the presence of sleep deprivation and sleep quality (Epworth Sleepiness Scale, Pittsburgh Sleep Quality Index), their cognitive status (Mini Mental State examination) and mood profiles (Profile of Mood States, POMS). Analysis of variance was used for comparison of questionnaires scores across specialties.

RESULTS

Authors reported no sleep deprivation, no sleep disorders and their outcomes, no changes in intellectual efficiency and no cognitive impairment in this population, neither in the areas performing night shifts nor in those involving only day shifts. Mood states measured by POMS showed a borderline level of Anger-Hostility (A) value among the residents of the medical area and services, and an increase slightly beyond the physiological levels of the T-score 50 of Fatigue-Inertia (F) always in the same groups. An increase in the Vigour-Activity (V) value beyond T-score 50 levels was also observed among residents of all the areas considered.

CONCLUSIONS

Emotional involvement could be attributed to the gap between high professional demand and lack of experience and knowledge among trainees. Tutors should help their students in order to identify earlier changes in the mood. Improvement in the organization of the trainee's activity could reduce the emotional overload.

Objective Assessment of Fitness to Perform (FTOP) After Surgical Night Shifts in the Netherlands: An Observational Study Using the Validated FTOP Self-test in Daily Surgical Practice

BACKGROUND

Surgical skills and decision making are influenced by alertness, reaction time, eye-hand coordination, and concentration. Night shift might impair these functions but it is unclear to what extent. The aim of this study was to investigate whether a night shift routinely impairs the surgeon's fitness to perform and whether this reaches a critical limit as compared to relevant frames of reference.

METHODS

Consultants (n = 59) and residents (n = 103) conducted fitness to perform measurements at precall, postcall, and noncall moments. This validated self-test consists of an adaptive tracker that is able to objectively measure alertness, reaction time, concentration, and eye-hand coordination, and multiple visual analog scales to subjectively score alertness. Results are compared to sociolegal (ethanol) and professional (operative skills) frames of reference that refer to a decrease under the influence of 0.06% ethanol.

RESULTS

Residents spent 1.7 call hours asleep on average as compared to 5.4 for consultants. Subjective alertness decreased in residents after night shifts (-13, P < 0.001) but not in consultants (-1.2, P = NS). The overnight difference in tracker score was -1.17 (P < 0.001) for residents and 0.46 (P = NS) for surgeons. Postcall subjective alertness only correlated to objective alertness in consultants. For residents, hours slept on-call correlated to objective alertness. For consultants, subsequent night calls significantly correlated to objective alertness, with the third subsequent call related to performance below the reference.

CONCLUSIONS

Consultants remain fit to perform after night call, but subsequent calls may compromise clinical activities. This study provides insight and awareness of individual performance with clear frames of reference.