Influence of simulation on electronic health record use patterns among pediatric residents

Clinical Information Retrieval: A Literature Review

Clinical information retrieval (IR) plays a vital role in modern healthcare by facilitating efficient access and analysis of medical literature for clinicians and researchers. This scoping review aims to offer a comprehensive overview of the current state of clinical IR research and identify gaps and potential opportunities for future studies in this field. The main objective was to assess and analyze the existing literature on clinical IR, focusing on the methods, techniques, and tools employed for effective retrieval and analysis of medical information. Adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we conducted an extensive search across databases such as Ovid Embase, Ovid Medline, Scopus, ACM Digital Library, IEEE Xplore, and Web of Science, covering publications from January 1, 2010, to January 4, 2023. The rigorous screening process led to the inclusion of 184 papers in our review. Our findings provide a detailed analysis of the clinical IR research landscape, covering aspects like publication trends, data sources, methodologies, evaluation metrics, and applications. The review identifies key research gaps in clinical IR methods such as indexing, ranking, and query expansion, offering insights and opportunities for future studies in clinical IR, thus serving as a guiding framework for upcoming research efforts in this rapidly evolving field. The study also underscores an imperative for innovative research on advanced clinical IR systems capable of fast semantic vector search and adoption of neural IR techniques for effective retrieval of information from unstructured electronic health records (EHRs).

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-024-00159-4.

Uses of Electronic Health Record Data to Measure the Clinical Learning Environment of Graduate Medical Education Trainees: A Systematic Review

PURPOSE

This study systematically reviews the uses of electronic health record (EHR) data to measure graduate medical education (GME) trainee competencies.

METHOD

In January 2022, the authors conducted a systematic review of original research in MEDLINE from database start to December 31, 2021. The authors searched for articles that used the EHR as their data source and in which the individual GME trainee was the unit of observation and/or unit of analysis. The database query was intentionally broad because an initial survey of pertinent articles identified no unifying Medical Subject Heading terms. Articles were coded and clustered by theme and Accreditation Council for Graduate Medical Education (ACGME) core competency.

RESULTS

The database search yielded 3,540 articles, of which 86 met the study inclusion criteria. Articles clustered into 16 themes, the largest of which were trainee condition experience (17 articles), work patterns (16 articles), and continuity of care (12 articles). Five of the ACGME core competencies were represented (patient care and procedural skills, practice-based learning and improvement, systems-based practice, medical knowledge, and professionalism). In addition, 25 articles assessed the clinical learning environment.

CONCLUSIONS

This review identified 86 articles that used EHR data to measure individual GME trainee competencies, spanning 16 themes and 6 competencies and revealing marked between-trainee variation. The authors propose a digital learning cycle framework that arranges sequentially the uses of EHR data within the cycle of clinical experiential learning central to GME. Three technical components necessary to unlock the potential of EHR data to improve GME are described: measures, attribution, and visualization. Partnerships between GME programs and informatics departments will be pivotal in realizing this opportunity.

Identification of robust deep neural network models of longitudinal clinical measurements

Deep learning (DL) from electronic health records holds promise for disease prediction, but systematic methods for learning from simulated longitudinal clinical measurements have yet to be reported. We compared nine DL frameworks using simulated body mass index (BMI), glucose, and systolic blood pressure trajectories, independently isolated shape and magnitude changes, and evaluated model performance across various parameters (e.g., irregularity, missingness). Overall, discrimination based on variation in shape was more challenging than magnitude. Time-series forest-convolutional neural networks (TSF-CNN) and Gramian angular field(GAF)-CNN outperformed other approaches (P < 0.05) with overall area-under-the-curve (AUCs) of 0.93 for both models, and 0.92 and 0.89 for variation in magnitude and shape with up to 50% missing data. Furthermore, in a real-world assessment, the TSF-CNN model predicted T2D with AUCs reaching 0.72 using only BMI trajectories. In conclusion, we performed an extensive evaluation of DL approaches and identified robust modeling frameworks for disease prediction based on longitudinal clinical measurements.

State of the evidence on simulation-based electronic health records training: A scoping review

This study synthesized the available evidence of simulation-based electronic health records (EHRs) training in educational and clinical environments for healthcare providers in the literature. The Arksey and O'Malley methodological framework was employed. A systematic search was carried out in relevant databases from inception to January 2020, identifying 24 studies for inclusion. Three themes emerged: (a) role of simulation-based EHR training in evaluating improvement interventions, (b) debriefing and feedback methods used, and (c) challenges of evaluating simulation-based EHR training. The majority of the studies aimed to emphasize the practical skills of individual medical trainees and employed post-simulation feedback as the feedback method. Future research should focus on (a) using simulation-based EHR training to achieve specific learning goals, (b) investigating aspects of clinical performance that are susceptible to skill decay, and (c) examining the influence of simulation-based EHR training on team dynamics.

Using a Semi-Structured Qualitative Interview to Evaluate Pediatric Interns' Use of the Electronic Medical Record in a Simulated Setting

Introduction

Effective use of electronic medical record (EMR) is paramount to delivering safe and effective care. Current EMR education is inadequate, with literature showing frequent deficiencies in skills needed to obtain and interpret data. This study aims to evaluate pediatric interns' perception of EMR inclusion in scenario-based simulation training.

Methods

A total of 13 pediatric interns participated in an EMR-enhanced, multidisciplinary simulation of a pediatric patient with septic shock during the 2019-2020 academic year. Following the simulation, the interns participated in a semi-structured interview to evaluate the experience of having the EMR incorporated into the simulation and what benefits it offers.

Results

Of the 13 interns, 12 (92%) felt that incorporating the EMR into the simulation increased the realism of the scenario. All (100%) interns reported that EMR inclusion led to increased learning about the EMR, including gaining or re-learning skills needed to access or interpret electronic clinical data. Participants felt that EMR inclusion in the simulation provided valuable learning opportunities not present in traditional EMR education.

Conclusions

Integrating the EMR into simulation is viewed positively by pediatric interns, is perceived to improve simulation realism, and helps teach important EMR skills. EMR training would benefit from incorporation into scenario-based simulations.

The Impact of Electronic Health Record-Based Simulation During Intern Boot Camp: Interventional Study

BACKGROUND

Accurate data retrieval is an essential part of patient care in the intensive care unit (ICU). The electronic health record (EHR) is the primary method for data storage and data review. We previously reported that residents participating in EHR-based simulations have varied and nonstandard approaches to finding data in the ICU, with subsequent errors in recognizing patient safety issues. We hypothesized that a novel EHR simulation-based training exercise would decrease EHR use variability among intervention interns, irrespective of prior EHR experience.

OBJECTIVE

This study aims to understand the impact of a novel, short, high-fidelity, simulation-based EHR learning activity on the intern data gathering workflow and satisfaction.

METHODS

A total of 72 internal medicine interns across the 2018 and 2019 academic years underwent a dedicated EHR training session as part of a week-long boot camp early in their training. We collected data on previous EHR and ICU experience for all subjects. Training consisted of 1 hour of guided review of a high-fidelity, simulated ICU patient chart focusing on best navigation practices for data retrieval. Specifically, the activity focused on using high- and low-yield data visualization screens determined by expert consensus. The intervention group interns then had 20 minutes to review a new simulated patient chart before the group review. EHR screen navigation was captured using screen recording software and compared with data from existing ICU residents performing the same task on the same medical charts (N=62). Learners were surveyed immediately and 6 months after the activity to assess satisfaction and preferred EHR screen use.

RESULTS

Participants found the activity useful and enjoyable immediately and after 6 months. Intervention interns used more individual screens than reference residents (18 vs 20; P=.008), but the total number of screens used was the same (35 vs 38; P=.30). Significantly more intervention interns used the 10 most common screens (73% vs 45%; P=.001). Intervention interns used high-yield screens more often and low-yield screens less often than the reference residents, which are persistent on self-report 6 months later.

CONCLUSIONS

A short, high-fidelity, simulation-based learning activity focused on provider-specific data gathering was found to be enjoyable and to modify navigation patterns persistently. This suggests that workflow-specific simulation-based EHR training throughout training is of educational benefit to residents.

Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods

OBJECTIVE

To systematically review published literature and identify consistency and variation in the aims, measures, and methods of studies using electronic health record (EHR) audit logs to observe clinical activities.

MATERIALS AND METHODS

In July 2019, we searched PubMed for articles using EHR audit logs to study clinical activities. We coded and clustered the aims, measures, and methods of each article into recurring categories. We likewise extracted and summarized the methods used to validate measures derived from audit logs and limitations discussed of using audit logs for research.

RESULTS

Eighty-five articles met inclusion criteria. Study aims included examining EHR use, care team dynamics, and clinical workflows. Studies employed 6 key audit log measures: counts of actions captured by audit logs (eg, problem list viewed), counts of higher-level activities imputed by researchers (eg, chart review), activity durations, activity sequences, activity clusters, and EHR user networks. Methods used to preprocess audit logs varied, including how authors filtered extraneous actions, mapped actions to higher-level activities, and interpreted repeated actions or gaps in activity. Nineteen studies validated results (22%), but only 9 (11%) through direct observation, demonstrating varying levels of measure accuracy.

DISCUSSION

While originally designed to aid access control, EHR audit logs have been used to observe diverse clinical activities. However, most studies lack sufficient discussion of measure definition, calculation, and validation to support replication, comparison, and cross-study synthesis.

CONCLUSION

EHR audit logs have potential to scale observational research but the complexity of audit log measures necessitates greater methodological transparency and validated standards.

Computer-based simulation to reduce EHR-related chemotherapy ordering errors

BACKGROUND

The electronic health record (EHR) is a contributor to serious patient harm occurring within a sociotechnical system. Chemotherapy ordering is a high-risk task due to the complex nature of ordering workflows and potential detrimental effects if wrong chemotherapeutic doses are administered. Many chemotherapy ordering errors cannot be mitigated through systems-based changes due to the limited extent to which individual institutions are able to customize proprietary EHR software. We hypothesized that simulation-based training could improve providers' ability to identify and mitigate common chemotherapy ordering errors.

METHODS

Pediatric hematology/oncology providers voluntarily participated in simulations using an EHR testing ("Playground") environment. The number of safety risks identified and mitigated by each provider at baseline was recorded. Risks were reviewed one-on-one after initial simulations and at a group "lunch-and-learn" session. At three-month follow-up, repeat simulations assessed for improvements in error identification and mitigation, and providers were surveyed about prevention of real-life safety events.

RESULTS

The 8 participating providers identified and mitigated an average of 5.5 out of 10 safety risks during the initial simulation, compared 7.4 safety risks at the follow up simulation (p=0.030). Two of the providers (25%) reported preventing at least one real-world patient safety event in the clinical setting as a result of the initial training session.

CONCLUSIONS

Simulation-based training may reduce providers' susceptibility to chemotherapy ordering safety vulnerabilities within the EHR. This approach may be used when systems-based EHR improvements are not feasible due to limited ability to customize local instances of proprietary EHR software.

Synthea™ Novel coronavirus (COVID-19) model and synthetic data set

March through May 2020, a model of novel coronavirus (COVID-19) disease progression and treatment was constructed for the open-source Synthea patient simulation. The model was constructed using three peer-reviewed publications published in the early stages of the global pandemic, when less was known, along with emerging resources, data, publications, and clinical knowledge. The simulation outputs synthetic Electronic Health Records (EHR), including the daily consumption of Personal Protective Equipment (PPE) and other medical devices and supplies.

For this simulation, we generated 124,150 synthetic patients, with 88,166 infections and 18,177 hospitalized patients. Patient symptoms, disease severity, and morbidity outcomes were calibrated using clinical data from the peer-reviewed publications. 4.1% of all simulated infected patients died and 20.6% were hospitalized. At peak observation, 548 dialysis machines and 209 mechanical ventilators were needed. This simulation and the resulting data have been used for the development of algorithms and prototypes designed to address the current or future pandemics, and the model can continue to be refined to incorporate emerging COVID-19 knowledge, variations in patterns of care, and improvement in clinical outcomes. The resulting model, data, and analysis are available as open-source code on GitHub and an open-access data set is available for download.

Education and Training on Electronic Medical Records (EMRs) for health care professionals and students: A Scoping Review

BACKGROUND AND OBJECTIVES

The ability of health care providers and students to use EMRs efficiently can lead to achieving improved clinical outcomes. Training policies and strategies play a major role in successful technology implementation and ongoing use of the EMR systems. To provide evidence-based guidance for developing and implementing educational interventions and training, we reviewed and summarized the current literature on EMR training targeting both healthcare professionals (HCP) and students.

METHODS

We used the Joanna Briggs Institute (JBI) approach for scoping reviews and the PRISMA extension of scoping reviews (PRISMA-ScR) checklist for reporting our review. 46 full-text articles that met the eligibility criteria were selected for the review. Narrative synthesis was performed to summarize the evidence using numerical and descriptive analysis. We used inductive content analysis for categorizing the training methods. Also, the modified version of the Kirkpatrick's levels model was used for abstracting the training outcome.

RESULTS

Five types of training methods were identified: one-on-one training, peer-coach training, classroom training (CRT), computer-based training (CBT), and blended training. A variety of CBT platforms were used, including a prototype academic electronic medical record system (AEMR), AEMR/simulated EMR (Sim-EMR), mobile based AEMR, eLearning, and electronic educational materials. Each training intervention could have resulted in several outcomes. Most outcomes were related to levels 1-3 of the Kirkpatrick model that involves learners (n = 108), followed by level 4a that involves organizations (n = 7), and lastly level 4b that involves patients (n = 1). The outcomes related to participants' knowledge (level 2b) was the most often measured training outcome (n = 44).

CONCLUSIONS

This review presents a comprehensive synthesis of the evidence on EMR training. A variety of training methods, participants, locations, strategies, and outcomes were described in the studies. Training should be aligned with the particular training needs, training objectives, EMR system utilized, and organizational environment. A training plan should include an overall goal and SMART (Specific, Measurable, Achievable, Realistic, Tangible) training objectives, that would allow a more rigorous evaluation of the training outcomes.

Using Clinical Simulations to Train Healthcare Professionals to Use Electronic Health Records: A Literature Review

Unintended consequences are adverse events directly related to information technology and may result from inappropriate use of electronic health records by healthcare professionals. Electronic health record competency training has historically used didactic lectures with hands-on experience in a live classroom, and this method fails to teach learners proficiency because the sociotechnical factors that are present in real-world settings are excluded. Additionally, on-the-job training to gain competency can impair patient safety because it distracts clinicians from patient care activities. Clinical simulation-based electronic health record training allows learners to acquire technical and nontechnical skills in a safe environment that will not compromise patient safety. The purpose of this literature review was to summarize the current state-of-the-science on the use of clinical simulations to train healthcare professionals to use electronic health records. The benefits of using simulation-based training that incorporates an organization's contextual factors include improvement of interdisciplinary team communication, clinical performance, clinician-patient-technology communication skills, and recognition of patient safety issues. Design considerations for electronic health record training using clinical simulations involve establishing course objectives, identifying outcome measures, establishing content requirements of both the clinical simulation and electronic health record, and providing adequate debriefing.

Educational approaches for improving physicians' use of health information technology

The benefits of Health Information Technology (HIT) depend on the way they are being used. Education and training are often needed to move from basic to advanced, value-adding, use. In this article, we describe three educational approaches that can help in achieving this goal: "productive failure," video tutorials, and simulation. We describe the rationale behind these approaches, their strengths, and limitations and illustrate their application, respectively, to three problems associated with the use of HIT in clinical practice: improving data quality within Electronic Medical Records (EMRs) at the point of data entry, use of advanced EMR features for chronic disease management, and impact of the EMR on patient-clinician communication. We conclude that, while these approaches are promising, there is a need for innovation and diversity of educational approaches to address use of advanced HIT features, identified challenges with HIT, and usage in context-as well as for rigorous evaluation.