A simple electronic medical record system designed for research

Clinician perspectives on how situational context and augmented intelligence design features impact perceived usefulness of sepsis prediction scores embedded within a simulated electronic health record

OBJECTIVE

Obtain clinicians' perspectives on early warning scores (EWS) use within context of clinical cases.

MATERIAL AND METHODS

We developed cases mimicking sepsis situations. De-identified data, synthesized physician notes, and EWS representing deterioration risk were displayed in a simulated EHR for analysis. Twelve clinicians participated in semi-structured interviews to ascertain perspectives across four domains: (1) Familiarity with and understanding of artificial intelligence (AI), prediction models and risk scores; (2) Clinical reasoning processes; (3) Impression and response to EWS; and (4) Interface design. Transcripts were coded and analyzed using content and thematic analysis.

RESULTS

Analysis revealed clinicians have experience but limited AI and prediction/risk modeling understanding. Case assessments were primarily based on clinical data. EWS went unmentioned during initial case analysis; although when prompted to comment on it, they discussed it in subsequent cases. Clinicians were unsure how to interpret or apply the EWS, and desired evidence on its derivation and validation. Design recommendations centered around EWS display in multi-patient lists for triage, and EWS trends within the patient record. Themes included a "Trust but Verify" approach to AI and early warning information, dichotomy that EWS is helpful for triage yet has disproportional signal-to-high noise ratio, and action driven by clinical judgment, not the EWS.

CONCLUSIONS

Clinicians were unsure of how to apply EWS, acted on clinical data, desired score composition and validation information, and felt EWS was most useful when embedded in multi-patient views. Systems providing interactive visualization may facilitate EWS transparency and increase confidence in AI-generated information.

Safety planning intervention and follow-up: A telehealth service model for suicidal individuals in emergency department settings: Study design and protocol

BACKGROUND

The Safety Planning Intervention with follow-up services (SPI+) is a promising suicide prevention intervention, yet many Emergency Departments (EDs) lack the resources for adequate implementation. Comprehensive strategies addressing structural and organizational barriers are needed to optimize SPI+ implementation and scale-up. This protocol describes a test of one strategy in which ED staff connect at-risk patients to expert clinicians from a Suicide Prevention Consultation Center (SPCC) via telehealth.

METHOD

This stepped wedge, cluster-randomized trial compares the effectiveness, implementation, cost, and cost offsets of SPI+ delivered by SPCC clinicians versus ED-based clinicians (enhanced usual care; EUC). Eight EDs will start with EUC and cross over to the SPCC phase. Blocks of two EDs will be randomly assigned to start dates 3 months apart. Approximately 13,320 adults discharged following a suicide-related ED visit will be included; EUC and SPCC samples will comprise patients from before and after SPCC crossover, respectively. Effectiveness data sources are electronic health records, administrative claims, and the National Death Index. Primary effectiveness outcomes are presence of suicidal behavior and number/type of mental healthcare visits and secondary outcomes include number/type of suicide-related acute services 6-months post-discharge. We will use the same data sources to assess cost offsets to gauge SPCC scalability and sustainability. We will examine preliminary implementation outcomes (reach, adoption, fidelity, acceptability, and feasibility) through patient, clinician, and health-system leader interviews and surveys.

CONCLUSION

If the SPCC demonstrates clinical effectiveness and health system cost reduction, it may be a scalable model for evidence-based suicide prevention in the ED.

Initial Development of an Automated Platform for Assessing Trainee Performance on Case Presentations

Background

Oral case presentation is a crucial skill of physicians and a key component of team-based care. However, consistent and objective assessment and feedback on presentations during training are infrequent.

Objective

To determine the potential value of applying natural language processing, computer software that extracts meaning from text, to transcripts of oral case presentations as a strategy to assess their quality automatically and objectively.

Methods

We transcribed a collection of simulated oral case presentations. The presentations were from eight critical care fellows and one critical care attending. They were instructed to review the medical charts of 11 real intensive care unit patient cases and to audio record themselves, presenting each case as if they were doing so on morning rounds. We then used natural language processing to convert the transcripts from human-readable text into machine-readable numbers. These numbers represent details of the presentation style and content. The distance between the numeric representation of two different transcripts negatively correlates with the similarity of those two transcripts. We ranked fellows on the basis of how similar their presentations were to the attending's presentations.

Results

The 99 presentations included 260 minutes of audio (mean length: 2.6 ± 1.24 min per case). On average, 23.88 ± 2.65 sentences were spoken, and each sentence had 14.10 ± 0.67 words, 3.62 ± 0.15 medical concepts, and 0.75 ± 0.09 medical adjectives. When ranking fellows on the basis of how similar their presentations were to the attending's presentation, we found a gap between the five fellows with the most similar presentations and the three fellows with the least similar presentations (average group similarity scores of 0.62 ± 0.01 and 0.53 ± 0.01, respectively). Rankings were sensitive to whether presentation style or content information were weighted more heavily when calculating transcript similarity.

Conclusion

Natural language processing enabled the ranking of case presentations on the basis of how similar they were to a reference presentation. Although additional work is needed to convert these rankings, and underlying similarity scores, into actionable feedback for trainees, these methods may support new tools for improving medical education.