Comparing the Impact of Double and Single Screen Electronic Health Records on Doctor-Patient Non-Verbal Communication

Automatic Classification of Screen Gaze and Dialogue in Doctor-Patient-Computer Interactions: Computational Ethnography Algorithm Development and Validation

Background

The study of doctor-patient-computer interactions is a key research area for examining doctor-patient relationships; however, studying these interactions is costly and obtrusive as researchers usually set up complex mechanisms or intrude on consultations to collect, then manually analyze the data.

Objective

We aimed to facilitate human-computer and human-human interaction research in clinics by providing a computational ethnography tool: an unobtrusive automatic classifier of screen gaze and dialogue combinations in doctor-patient-computer interactions.

Methods

The classifier’s input is video taken by doctors using their computers' internal camera and microphone. By estimating the key points of the doctor's face and the presence of voice activity, we estimate the type of interaction that is taking place. The classification output of each video segment is 1 of 4 interaction classes: (1) screen gaze and dialogue, wherein the doctor is gazing at the computer screen while conversing with the patient; (2) dialogue, wherein the doctor is gazing away from the computer screen while conversing with the patient; (3) screen gaze, wherein the doctor is gazing at the computer screen without conversing with the patient; and (4) other, wherein no screen gaze or dialogue are detected. We evaluated the classifier using 30 minutes of video provided by 5 doctors simulating consultations in their clinics both in semi- and fully inclusive layouts.

Results

The classifier achieved an overall accuracy of 0.83, a performance similar to that of a human coder. Similar to the human coder, the classifier was more accurate in fully inclusive layouts than in semi-inclusive layouts.

Conclusions

The proposed classifier can be used by researchers, care providers, designers, medical educators, and others who are interested in exploring and answering questions related to screen gaze and dialogue in doctor-patient-computer interactions.

Augmenting patient safety through participation by design - An assessment of dual monitors for patients in the outpatient clinic

BACKGROUND

Patients and physicians engaging together in the electronic health record (EHR) during clinical visits may provide opportunities to both improve patient understanding and reduce medical errors.

OBJECTIVE

To assess the potential impact of a patient EHR display intervention on patient quality and safety. We hypothesized that if patients had a dedicated display with an explicit invitation to follow clinicians in the EHR that this would identify several opportunities to engage patients in their care quality and safety.

MATERIAL AND METHODS

Physician-patient outpatient encounters (24 patients and 8 physicians) were videotaped. Encounters took place in a hospital-based general internal medicine outpatient clinic where physicians and patients had their respective EHR monitors. Following the visits, each patient and physician was interviewed for 30 min to understand their perception of the mirrored-screen setting.

RESULTS

The following 7 themes were identified (a) curiosity, (b) opportunity to ask questions, (c) error identification, (d) control over medications, (e) awareness, (f) shared understanding & decision-making, (g) data privacy. These themes collectively comprised a conceptual model for how patient engagement in electronic health record use, through a dedicated second screen or an explicitly shared screen, relates to safety and quality opportunities. Therefore, the double EHR screen provides an explicit invitation for patients to join the process to influence safety.

CONCLUSION

Desired outcomes include real-time error identification and better-shared understanding and decision-making, leading to better downstream follow-through with care plans.