A piece of my mind. Copy-and-paste

Assessing the Impact of a Note-Writing Session and Standardized Note Template on Medical Student Note Length and Quality

PURPOSE

To determine whether a formal note-writing session and note template for medical students (MS) during the Core Clerkship in Pediatrics (CCP) increase note quality, shortens note length, and decreases time of documentation.

METHODS

In this single site, prospective study, MS participating in an 8-week CCP received a didactic session on note-writing in the electronic health record (EHR) and utilized EHR template developed for the study. We assessed note quality (measured by Physician Documentation Quality Instrument-9 [PDQI-9]), note length and note documentation time in this group compared to MS notes on the CCP in the prior academic year. We used descriptive statistics and Kruskal-Wallis tests for analysis.

RESULTS

We analyzed 121 notes written by 40 students in the control group and 92 notes writing by 41 students in the intervention group. Notes from the intervention group were more "up to date," "accurate," "organized," and "comprehensible" compared to the control group (P = 0.02, P = 0.04, P = 0.01, and P = 0.02, respectively). Intervention group notes received higher cumulative PDQI-9 scores compared to the control group (median score 38 (IQR 34-42) versus 36 (IQR 32-40) out of 45 total, P = 0.04). Intervention group notes were approximately 35% shorter than the control group notes (median 68.5 lines vs 105 lines, P < 0.0001) and were signed earlier than control group notes (median file time 316 minute vs 352 minute, P = 0.02).

CONCLUSIONS

The intervention successfully decreased note length, improved note quality based on standardized metrics, and reduced time to completion of note documentation.

Impact of clinical note format on diagnostic accuracy and efficiency

BACKGROUND

Clinician notes are structured in a variety of ways. This research pilot tested an innovative study design and explored the impact of note formats on diagnostic accuracy and documentation review time.

OBJECTIVE

To compare two formats for clinical documentation (narrative format vs. list of findings) on clinician diagnostic accuracy and documentation review time.

METHOD

Participants diagnosed written clinical cases, half in narrative format, and half in list format. Diagnostic accuracy (defined as including correct case diagnosis among top three diagnoses) and time spent processing the case scenario were measured for each format. Generalised linear mixed regression models and bias-corrected bootstrap percentile confidence intervals for mean paired differences were used to analyse the primary research questions.

RESULTS

Odds of correctly diagnosing list format notes were 26% greater than with narrative notes. However, there is insufficient evidence that this difference is significant (75% CI 0.8-1.99). On average the list format notes required 85.6 more seconds to process and arrive at a diagnosis compared to narrative notes (95% CI -162.3, -2.77). Of cases where participants included the correct diagnosis, on average the list format notes required 94.17 more seconds compared to narrative notes (75% CI -195.9, -8.83).

CONCLUSION

This study offers note format considerations for those interested in improving clinical documentation and suggests directions for future research. Balancing the priority of clinician preference with value of structured data may be necessary.

IMPLICATIONS

This study provides a method and suggestive results for further investigation in usability of electronic documentation formats.

Scoping review: Positive and negative impact of technology on clinicians

BACKGROUND

Unnecessary electronic health record (EHRs) documentation burden and usability issues have negatively impacted clinician well-being (e.g., burnout and moral distress).

PURPOSE

This scoping review was conducted by members from three expert panels of the American Academy of Nurses to generate consensus on the evidence of both positive and negative impact of EHRs on clinicians.

METHODS

The scoping review was conducted based on Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Extension for Scoping Reviews guidelines.

RESULTS

The scoping review captured 1,886 publications screened against title and abstract 1,431 excluded, examined 448 in a full-text review, excluded 347 with 101 studies informing the final review.

DISCUSSION

Findings suggest few studies that have explored the positive impact of EHRs and more studies that have explored the clinician's satisfaction and work burden. Significant gaps were identified in associating distress to use of EHRs and minimal studies on EHRs' impact on nurses.

CONCLUSION

Examined the evidence of HIT's positive and negative impacts on clinician's practice, clinicians work environment, and if psychological impact differed among clinicians.

Medical Records: A Historical Narrative

The history of medical records is thousand-year-long, with earlier roots in ancient civilizations. Until the 19th century, medical records mainly served educational purposes, later assuming other roles such as in insurance or legal procedures. This article comprehensively describes and reviews the development of medical records from ancient to modern times in Europe and North America, reflecting alterations and adaptations compliant with the mental and technological capabilities of a given period. We searched PubMed and Google Scholar databases to collect pertinent articles. English articles or those having English abstracts were considered. The search terms included “Medical Records,” “Health Records,” “History of Medicine,” and “eHealth” and covered the last hundred years. References were also picked out from the identified articles. Overall, 600 articles were identified, 158 of which were judged thematically relevant. The general conclusion is that medical records undergo a revolutionary change from paper-based to electronic format, which reflects the development of eHealth systems. The migration process to eHealth records involves the use of artificial intelligence (AI) algorithms that streamline medical services by using faster and simpler working methods. AI benefits both patients and providers as it improves patient management and communication among medical centers, spares resources, identifies contamination or infections, and limits health costs. These advantages have become pointedly apparent during the recent COVID-19 scourge.

Prevalence and Sources of Duplicate Information in the Electronic Medical Record

IMPORTANCE

Duplicated text is a well-documented hazard in electronic medical records (EMRs), leading to wasted clinician time, medical error, and burnout. This study hypothesizes that text duplication is prevalent and increases with time and EMR size and that duplicate information is shared across authors.

OBJECTIVE

To examine the prevalence and scope of duplication behavior in clinical notes from a large academic health system and the factors associated with duplication.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective, cross-sectional analysis of note length and content duplication rates used a set of 10 adjacent word tokens (ie, a 10-gram) sliding-window approach to identify spans of text duplicated exactly from earlier notes in a patient's record for all inpatient and outpatient notes written within the University of Pennsylvania Health System from January 1, 2015, through December 31, 2020. Text duplicated from a different author vs text duplicated from the same author was quantified. Furthermore, novel text and duplicated text per author for various note types and author types, as well as per patient record by number of notes in the record, were quantified. Information scatter, another documentation hazard, was defined as the inverse of novel text per note, and the association between information duplication and information scatter was graphed. Data analysis was performed from January to March 2022.

MAIN OUTCOMES AND MEASURES

Total, novel, and duplicate text by note type and note author were determined, as were the mean intra-author and inter-author duplication per note by type and author.

RESULTS

There were a total of 104 456 653 notes for 1 960 689 unique patients consisting of 32 991 489 889 words; 50.1% of the total text in the record (16 523 851 210 words) was duplicated from prior text written about the same patient. The duplication fraction increased year-over-year, from 33.0% for notes written in 2015 to 54.2% for notes written in 2020. Of the text duplicated, 54.1% came from text written by the same author, whereas 45.9% was duplicated from a different author. Records with more notes had more total duplicate text, approaching 60%. Note types with high information scatter tended to have low information overload, and vice versa, suggesting a trade-off between these 2 hazards under the current documentation paradigm.

CONCLUSIONS AND RELEVANCE

Duplicate text casts doubt on the veracity of all information in the medical record, making it difficult to find and verify information in day-to-day clinical work. The findings of this cross-sectional study suggest that text duplication is a systemic hazard, requiring systemic interventions to fix, and simple solutions such as banning copy-paste may have unintended consequences, such as worsening information scatter. The note paradigm should be further examined as a major cause of duplication and scatter, and alternative paradigms should be evaluated.

Analysis of a Near Peer Tutoring Program to Improve Medical Students' Note Writing Skills

Problem: The ability to document a patient encounter is integral for any physician. Previous studies indicate that medical students' note writing skills are poor due to a lack of formal clinical documentation instruction. Barriers to formally teaching students how to write patient notes include the significant time burden and variability in faculty feedback. Implementing a near-peer teaching program to teach students how to write a patient note can overcome these barriers and fill this curricular gap. Intervention: Fourth year medical students who completed a Teaching in Medicine elective course were trained to provide individual feedback to 1^st and 2^nd year students on note writing as a part of their Introduction to Clinical Medicine course. In order to determine the effect of this two year, near-peer feedback program on note writing skills, we analyzed students' scores on the note writing portion of two Objective Structured Clinical Exams that took place at the end of the 2^nd and 3^rd years of medical school. Context: The near-peer feedback sessions were implemented in the Fall of 2013 during the preclinical years of the medical school curriculum. Data from students who received near-peer feedback (N = 112) were compared to a historical control group who did not receive near-peer feedback on their note writing (N = 110). Objective Structured Clinical Exam scores that were specific to note writing skills, including the history, physical exam, and differential diagnosis subscales were examined. Impact: The near-peer feedback had a positive impact on the quality of patient notes. On the end of the 2^nd year Objective Structured Clinical Exam, the near-peer feedback intervention group outperformed the no feedback group on the history and physical exam subscale scores but not on the differential diagnosis subscale score. One year later, the near-peer feedback intervention group continued to outperform the no feedback group on the physical exam subscale score, but not the history or the differential diagnosis subscale scores. Lessons Learned: Near-peer teaching improves student documentation of the history and physical exam, however only the effects on the physical exam portion persist into the clinical years of training. Writing up a differential diagnosis is a skill that develops through the clerkship experience regardless of exposure to feedback in the preclinical years. Implementing near-peer teaching in the medical school setting is feasible and can provide students with valuable learning experiences without relying on clinical faculty.

"Note Bloat" impacts deep learning-based NLP models for clinical prediction tasks

One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates "bloated" notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts.

Using Technology to Enhance Communication

Digital communication, facilitated by the rise of the electronic health record and telehealth, has transformed clinical workflow. The communication tools, and the purposes they are being used for, need to account for the benefits, risks, and fault tolerance for each tool. In this article, the authors offer several suggestions on how to approach these important issues. These new digital communication tools open the door to novel care models for connecting patients and providers. Most importantly, the way a message is delivered, not the medium through which it is transmitted, is the key to successful communication.

Incorporating Domain Knowledge Into Language Models by Using Graph Convolutional Networks for Assessing Semantic Textual Similarity: Model Development and Performance Comparison

BACKGROUND

Although electronic health record systems have facilitated clinical documentation in health care, they have also introduced new challenges, such as the proliferation of redundant information through the use of copy and paste commands or templates. One approach to trimming down bloated clinical documentation and improving clinical summarization is to identify highly similar text snippets with the goal of removing such text.

OBJECTIVE

We developed a natural language processing system for the task of assessing clinical semantic textual similarity. The system assigns scores to pairs of clinical text snippets based on their clinical semantic similarity.

METHODS

We leveraged recent advances in natural language processing and graph representation learning to create a model that combines linguistic and domain knowledge information from the MedSTS data set to assess clinical semantic textual similarity. We used bidirectional encoder representation from transformers (BERT)-based models as text encoders for the sentence pairs in the data set and graph convolutional networks (GCNs) as graph encoders for corresponding concept graphs that were constructed based on the sentences. We also explored techniques, including data augmentation, ensembling, and knowledge distillation, to improve the model's performance, as measured by the Pearson correlation coefficient (r).

RESULTS

Fine-tuning the BERT_base and ClinicalBERT models on the MedSTS data set provided a strong baseline (Pearson correlation coefficients: 0.842 and 0.848, respectively) compared to those of the previous year's submissions. Our data augmentation techniques yielded moderate gains in performance, and adding a GCN-based graph encoder to incorporate the concept graphs also boosted performance, especially when the node features were initialized with pretrained knowledge graph embeddings of the concepts (r=0.868). As expected, ensembling improved performance, and performing multisource ensembling by using different language model variants, conducting knowledge distillation with the multisource ensemble model, and taking a final ensemble of the distilled models further improved the system's performance (Pearson correlation coefficients: 0.875, 0.878, and 0.882, respectively).

CONCLUSIONS

This study presents a system for the MedSTS clinical semantic textual similarity benchmark task, which was created by combining BERT-based text encoders and GCN-based graph encoders in order to incorporate domain knowledge into the natural language processing pipeline. We also experimented with other techniques involving data augmentation, pretrained concept embeddings, ensembling, and knowledge distillation to further increase our system's performance. Although the task and its benchmark data set are in the early stages of development, this study, as well as the results of the competition, demonstrates the potential of modern language model-based systems to detect redundant information in clinical notes.

Assessing the quality of electronic medical records as a platform for resident education

BACKGROUND

Previous studies have assessed note quality and the use of electronic medical record (EMR) as a part of medical training. However, a generalized and user-friendly note quality assessment tool is required for quick clinical assessment. We held a medical record writing competition and developed a checklist for assessing the note quality of participants' medical records. Using the checklist, this study aims to explore note quality between residents of different specialties and offer pedagogical implications.

METHODS

The authors created an inpatient checklist that examined fundamental EMR requirements through six note types and twenty items. A total of 149 records created by residents from 32 departments/stations were randomly selected. Seven senior physicians rated the EMRs using a checklist. Medical records were grouped as general medicine, surgery, paediatric, obstetrics and gynaecology, and other departments. The overall and group performances were analysed using analysis of variance (ANOVA).

RESULTS

Overall performance was rated as fair to good. Regarding the six note types, discharge notes (0.81) gained the highest scores, followed by admission notes (0.79), problem list (0.73), overall performance (0.73), progress notes (0.71), and weekly summaries (0.66). Among the five groups, other departments (80.20) had the highest total score, followed by obstetrics and gynaecology (78.02), paediatrics (77.47), general medicine (75.58), and surgery (73.92).

CONCLUSIONS

This study suggested that duplication in medical notes and the documentation abilities of residents affect the quality of medical records in different departments. Further research is required to apply the insights obtained in this study to improve the quality of notes and, thereby, the effectiveness of resident training.

Estimating redundancy in clinical text

The current mode of use of Electronic Health Records (EHR) elicits text redundancy. Clinicians often populate new documents by duplicating existing notes, then updating accordingly. Data duplication can lead to propagation of errors, inconsistencies and misreporting of care. Therefore, measures to quantify information redundancy play an essential role in evaluating innovations that operate on clinical narratives. This work is a quantitative examination of information redundancy in EHR notes. We present and evaluate two methods to measure redundancy: an information-theoretic approach and a lexicosyntactic and semantic model. Our first measure trains large Transformer-based language models using clinical text from a large openly available US-based ICU dataset and a large multi-site UK based Hospital. By comparing the information-theoretic efficient encoding of clinical text against open-domain corpora, we find that clinical text is ∼1.5× to ∼3× less efficient than open-domain corpora at conveying information. Our second measure, evaluates automated summarisation metrics Rouge and BERTScore to evaluate successive note pairs demonstrating lexicosyntactic and semantic redundancy, with averages from ∼43 to ∼65%.

Frequent but fragmented: use of note templates to document outpatient visits at an academic health center

Recent changes to billing policy have reduced documentation requirements for outpatient notes, providing an opportunity to rethink documentation workflows. While many providers use templates to write notes-whether to insert short phrases or draft entire notes-we know surprisingly little about how these templates are used in practice. In this retrospective cross-sectional study, we observed the templates that primary providers and other members of the care team used to write the provider progress note for 2.5 million outpatient visits across 52 specialties at an academic health center between 2018 and 2020. Templates were used to document 89% of visits, with a median of 2 used per visit. Only 17% of the 100 230 unique templates were ever used by more than one person and most providers had their own full-note templates. These findings suggest template use is frequent but fragmented, complicating template revision and maintenance. Reframing template use as a form of computer programming suggests ways to maintain the benefits of personalization while leveraging standardization to reduce documentation burden.

Writing Practices Associated With Electronic Progress Notes and the Preferences of Those Who Read Them: Descriptive Study

Background

Hospital progress notes can serve as an important communication tool. However, they are criticized for their length, preserved content, and for the time physicians spend writing them.

Objective

We aimed to describe hospital progress note content, writing and reading practices, and the preferences of those who create and read them prior to the implementation of a new electronic health record system.

Methods

Using a sample of hospital progress notes from 1000 randomly selected admissions, we measured note length, similarity of content in successive daily notes for the same patient, the time notes were signed and read, and who read them. We conducted focus group sessions with note writers, readers, and clinical leaders to understand their preferences.

Results

We analyzed 4938 inpatient progress notes from 418 authors. The average length was 886 words, and most were in the Assessment & Plan note section. A total of 29% of notes (n=1432) were signed after 4 PM. Notes signed later in the day were read less often. Notes were highly similar from one day to the next, and 26% (23/88) had clinical risk associated with the preserved content. Note content of the highest value varied according to the reader’s professional role.

Conclusions

Progress note length varied widely. Notes were often signed late in the day when they were read less often and were highly similar to the note from the previous day. Measuring note length, signing time, when and by whom notes are read, and the amount and safety of preserved content will be useful metrics for measuring how the new electronic health record system is used, and can aid improvements.

Length and Redundancy of Outpatient Progress Notes Across a Decade at an Academic Medical Center

IMPORTANCE

There is widespread concern that clinical notes have grown longer and less informative over the past decade. Addressing these concerns requires a better understanding of the magnitude, scope, and potential causes of increased note length and redundancy.

OBJECTIVE

To measure changes between 2009 and 2018 in the length and redundancy of outpatient progress notes across multiple medical specialties and investigate how these measures associate with author experience and method of note entry.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study was conducted at Oregon Health & Science University, a large academic medical center. Participants included clinicians and staff who wrote outpatient progress notes between 2009 and 2018 for a random sample of 200 000 patients. Statistical analysis was performed from March to August 2020.

EXPOSURES

Use of a comprehensive electronic health record to document patient care.

MAIN OUTCOMES AND MEASURES

Note length, note redundancy (ie, the proportion of text identical to the patient's last note), and percentage of templated, copied, or directly typed note text.

RESULTS

A total of 2 704 800 notes written by 6228 primary authors across 46 specialties were included in this study. Median note length increased 60.1% (99% CI, 46.7%-75.2%) from a median of 401 words (interquartile range [IQR], 225-660 words) in 2009 to 642 words (IQR, 399-1007 words) in 2018. Median note redundancy increased 10.9 percentage points (99% CI, 7.5-14.3 percentage points) from 47.9% in 2009 to 58.8% in 2018. Notes written in 2018 had a mean value of just 29.4% (99% CI, 28.2%-30.7%) directly typed text with the remaining 70.6% of text being templated or copied. Mixed-effect linear models found that notes with higher proportions of templated or copied text were significantly longer and more redundant (eg, in the 2-year model, each 1% increase in the proportion of copied or templated note text was associated with 1.5% [95% CI, 1.5%-1.5%] and 1.6% [95% CI, 1.6%-1.6%] increases in note length, respectively). Residents and fellows also wrote significantly (26.3% [95% CI, 25.8%-26.7%]) longer notes than more senior authors, as did more recent hires (1.8% for each year later [95% CI, 1.3%-2.4%]).

CONCLUSIONS AND RELEVANCE

In this study, outpatient progress notes grew longer and more redundant over time, potentially limiting their use in patient care. Interventions aimed at reducing outpatient progress note length and redundancy may need to simultaneously address multiple factors such as note template design and training for both new and established clinicians.

What's in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarization

Summarization of clinical narratives is a long-standing research problem. Here, we introduce the task of hospital-course summarization. Given the documentation authored throughout a patient's hospitalization, generate a paragraph that tells the story of the patient admission. We construct an English, text-to-text dataset of 109,000 hospitalizations (2M source notes) and their corresponding summary proxy: the clinician-authored "Brief Hospital Course" paragraph written as part of a discharge note. Exploratory analyses reveal that the BHC paragraphs are highly abstractive with some long extracted fragments; are concise yet comprehensive; differ in style and content organization from the source notes; exhibit minimal lexical cohesion; and represent silver-standard references. Our analysis identifies multiple implications for modeling this complex, multi-document summarization task.

Teasing out Artificial Intelligence in Medicine: An Ethical Critique of Artificial Intelligence and Machine Learning in Medicine

The rapid adoption and implementation of artificial intelligence in medicine creates an ontologically distinct situation from prior care models. There are both potential advantages and disadvantages with such technology in advancing the interests of patients, with resultant ontological and epistemic concerns for physicians and patients relating to the instatiation of AI as a dependent, semi- or fully-autonomous agent in the encounter. The concept of libertarian paternalism potentially exercised by AI (and those who control it) has created challenges to conventional assessments of patient and physician autonomy. The unclear legal relationship between AI and its users cannot be settled presently, an progress in AI and its implementation in patient care will necessitate an iterative discourse to preserve humanitarian concerns in future models of care. This paper proposes that physicians should neither uncritically accept nor unreasonably resist developments in AI but must actively engage and contribute to the discourse, since AI will affect their roles and the nature of their work. One's moral imaginative capacity must be engaged in the questions of beneficence, autonomy, and justice of AI and whether its integration in healthcare has the potential to augment or interfere with the ends of medical practice.

Effect of Outpatient Note Templates on Note Quality: NOTE (Notation Optimization through Template Engineering) Randomized Clinical Trial

BACKGROUND

This is the first randomized controlled trial evaluating the impact of note template design on note quality using a simulated patient encounter and a validated assessment tool.

OBJECTIVE

To compare note quality between two different templates using a novel randomized clinical simulation process.

DESIGN

A randomized non-blinded controlled trial of a standard note template versus redesigned template.

PARTICIPANTS

PGY 1-3 IM residents.

INTERVENTIONS

Residents documented the simulated patient encounter using one of two templates. The standard template was modeled after the usual outpatient progress note. The new template placed the assessment and plan section in the beginning, grouped subjective data into the assessment, and deemphasized less useful elements.

MAIN MEASURES

Note length; time to note completion; note template evaluation by resident authors; note evaluation by faculty reviewers.

KEY RESULTS

36 residents participated, 19 randomized to standard template, 17 to new. New template generated shorter notes (103 vs 285 lines, p < 0.001) that took the same time to complete (19.8 vs 21.6 min, p = 0.654). Using a 5-point Likert scale, residents considered new notes to have increased visual appeal (4 vs 3, p = 0.05) and less redundancy and clutter (4 vs 3, p = 0.006). Overall template satisfaction was not statistically different. Faculty reviewers rated the standard note more up-to-date (4.3 vs 2.7, p = 0.001), accurate (3.9 vs 2.6, p = 0.003), and useful (4 vs 2.8, p = 0.002), but less organized (3.3 vs 4.5, p < 0.001). Total quality was not statistically different.

CONCLUSIONS

Residents rated the new note template more visually appealing, shorter, and less cluttered. Faculty reviewers rated both note types equivalent in the overall quality but rated new notes inferior in terms of accuracy and usefulness though better organized. This study demonstrates a novel method of a simulated clinical encounter to evaluate note templates before the introduction into practice.

TRIAL REGISTRATION

ClinicalTrials.gov ID: NCT04333238.

A patient and family reporting system for perceived ambulatory note mistakes: experience at 3 U.S. healthcare centers

OBJECTIVE

The study sought to test a patient and family online reporting system for perceived ambulatory visit note inaccuracies.

MATERIALS AND METHODS

We implemented a patient and family electronic reporting system at 3 U.S. healthcare centers: a northeast urban academic adult medical center (AD), a northeast urban academic pediatric medical center (PED), and a southeast nonprofit hospital network (NET). Patients and families reported potential documentation inaccuracies after reading primary care and subspecialty visit notes. Results were characterized using descriptive statistics and coded for clinical relevance.

RESULTS

We received 1440 patient and family reports (780 AD, 402 PED, and 258 NET), and 27% of the reports identified a potential inaccuracy (25% AD, 35% PED, 28% NET). Among these, patients and families indicated that the potential inaccuracy was important or very important in 58% of reports (55% AD, 55% PED, 71% NET). The most common types of potential inaccuracies included description of symptoms (21%), past medical problems (21%), medications (18%), and important information that was missing (15%). Most patient- and family-reported inaccuracies resulted in a change to care or to the medical record (55% AD, 67% PED, data not available at NET).

DISCUSSION

About one-quarter of patients and families using an online reporting system identified potential documentation inaccuracies in visit notes and more than half were considered important by patients and clinicians, underscoring the potential role of patients and families as ambulatory safety partners.

CONCLUSIONS

Partnering with patients and families to obtain reports on inaccuracies in visit notes may contribute to safer care. Mechanisms to encourage greater use of patient and family reporting systems are needed.

Electronic Documentation in Pediatrics: The Rationale and Functionality Requirements

Clinical documentation has dramatically changed since the implementation and use of electronic health records and electronic provider documentation. The purpose of this report is to review these changes and promote the development of standards and best practices for electronic documentation for pediatric patients. In this report, we evaluate the unique aspects of clinical documentation for pediatric care, including specialized information needs and stakeholders specific to the care of children. Additionally, we explore new models of documentation, such as shared documentation, in which patients may be both authors and consumers, and among care teams while still maintaining the ability to clearly define care and services provided to patients in a given day or encounter. Finally, we describe alternative documentation techniques and newer technologies that could improve provider efficiency and the reuse of clinical data.

Research and Reporting Considerations for Observational Studies Using Electronic Health Record Data

Electronic health records (EHRs) are an increasingly important source of real-world health care data for observational research. Analyses of data collected for purposes other than research require careful consideration of data quality as well as the general research and reporting principles relevant to observational studies. The core principles for observational research in general also apply to observational research using EHR data, and these are well addressed in prior literature and guidelines. This article provides additional recommendations for EHR-based research. Considerations unique to EHR-based studies include assessment of the accuracy of computer-executable cohort definitions that can incorporate unstructured data from clinical notes and management of data challenges, such as irregular sampling, missingness, and variation across time and place. Principled application of existing research and reporting guidelines alongside these additional considerations will improve the quality of EHR-based observational studies.

[Legitimate resistance without technophobia: Analysis of electronic medical records impacts on the medical profession]

The objective of this short narrative literature review is to highlight the different difficulties encountered by medical doctor in the daily use of EMR. We show that these are not simple transitional phenomena related to a "resistance to change", but rather the fact of a deeper and unfinished transformation. Beyond the "perception of misfit with work processes" or the threat of a loss of autonomy, we propose to analyze this so-called "resistance" in relation to the formalization of medical work induced by EMR. Our question concerns the compatibility of the multiple objectives of EMR, the potential influence of computerization on the steps of entering and consulting medical information, the impact on the clinical reasoning, the reality of assistance to medical "performance". The question is not so much what EMRs do less well than the paper record, but to provide insights into how tomorrow's EMRs will do better than today's.

Improved efficiency of patient admission with electronic health records in neurosurgery

BACKGROUND

Electronic health records (EHRs) may be controversial but they have the potential to improve patient care. We investigated whether the introduction of an electronic template-based admission form for the collection of information about the patient's medical history and neurological and clinical state at admission in the neurosurgical unit might have an impact on the quality of documentation in a discharge record and the amount of time taken to produce this documentation.

METHOD

A new digital template-based admission form (EHR) was developed and assessed with QNOTE, an assessment tool of medical notes with standardised criteria and the possibility to benchmark the quality of documentations. This was compared to 30 prior paper-based handwritten documentations (HWD) regarding the utilisation of these medical notes for dictation of medical discharge records.

RESULTS

Implementation of the EHR significantly improved the quality of patient admission documentation with a QNOTE mean grand score of 87 ± 22 (p < 0.0001) compared to prior HWD with 44 ± 30. The mean documentation time for HWD was 8.1 min ± 4.1 min and the dictation time for discharge records was 10.6 min ± 3.5 min. After implementation of EHR, the documentation time increased slightly to 9.6 min ± 2.3 min (n.s.), while the time for dictation of discharge records was reduced to 5.1 min ± 1.2 min (p < 0.0001). There was a clear correlation between a higher quality of documentation and a higher needed documentation time as well as higher quality of documentation and lower dictation times of discharge records.

CONCLUSION

Implementation of the EHR improved the quality of patient admission documentation and reduced the dictation time of discharge records.

IMPLICATIONS

It is crucial to involve stakeholders and users of EHRs in a timely manner during the stage of development and implementation phase to ensure optimal results and better usability.

Clinical Documentation as End-User Programming

As healthcare providers have transitioned from paper to electronic health records they have gained access to increasingly sophisticated documentation aids such as custom note templates. However, little is known about how providers use these aids. To address this gap, we examine how 48 ophthalmologists and their staff create and use content-importing phrases - a customizable and composable form of note template - to document office visits across two years. In this case study, we find 1) content-importing phrases were used to document the vast majority of visits (95%), 2) most content imported by these phrases was structured data imported by data-links rather than boilerplate text, and 3) providers primarily used phrases they had created while staff largely used phrases created by other people. We conclude by discussing how framing clinical documentation as end-user programming can inform the design of electronic health records and other documentation systems mixing data and narrative text.

Electronic Health Records in Ophthalmology: Source and Method of Documentation

PURPOSE

This study analyzed and quantified the sources of electronic health record (EHR) text documentation in ophthalmology progress notes.

DESIGN

EHR documentation review and analysis.

METHODS

Setting: a single academic ophthalmology department.

STUDY POPULATION

a cohort study conducted between November 1, 2016, and December 31, 2018, using secondary EHR data and a follow-up manual review of a random samples. The cohort study included 123,274 progress notes documented by 42 attending providers. These notes were for patients with the 5 most common primary International Statistical Classification of Diseases and Related Health Problems, version 10, parent codes for each provider. For the manual review, 120 notes from 8 providers were randomly sampled. Main outcome measurements were characters or number of words in each note categorized by attribution source, author type, and time of creation.

RESULTS

Imported text entries made up the majority of text in new and return patients, 2,978 characters (77%) and 3,612 characters (91%). Support staff members authored substantial portions of notes; 3,024 characters (68%) of new patient notes, 3,953 characters (83%) of return patient notes. Finally, providers completed large amounts of documentation after clinical visits: 135 words (35%) of new patient notes, 102 words (27%) of return patient notes.

CONCLUSIONS

EHR documentation consists largely of imported text, is often authored by support staff, and is often written after the end of a visit. These findings raise questions about documentation accuracy and utility and may have implications for quality of care and patient-provider relationships.

Copy-and-Paste in Medical Student Notes: Extent, Temporal Trends, and Relationship to Scholastic Performance

BACKGROUND

Medical students may observe and subsequently perpetuate redundancy in clinical documentation, but the degree of redundancy in student notes and whether there is an association with scholastic performance are unknown.

OBJECTIVES

This study sought to quantify redundancy, defined generally as the proportion of similar text between two strings, in medical student notes and evaluate the relationship between note redundancy and objective indicators of student performance.

METHODS

Notes generated by medical students rotating through their medicine clerkship during a single academic year at our institution were analyzed. A student-patient interaction (SPI) was defined as a history and physical and at least two contiguous progress notes authored by the same student during a single patient's hospitalization. For some students, SPI pairs were available from early and late in the clerkship. Redundancy between analogous sections of consecutive notes was calculated on a 0 to 100% scale and was derived from edit distance, the number of changes needed to transform one text string into another. Indicators of student performance included United States Medical Licensing Exam (USMLE) scores.

RESULTS

Ninety-four single SPIs and 58 SPI pairs were analyzed. Redundancy in the assessment/plan section was high (40%) and increased within individual SPIs (to 60%; p < 0.001) and between SPI pairs over the course of the clerkship (by 30-40%; p < 0.001). Students in the lowest tertile of USMLE step II clinical knowledge scores had higher redundancy in the assessment/plan section than their classmates (67 ± 24% vs. 38 ± 22%; p = 0.002).

CONCLUSION

During the medicine clerkship, the assessment/plan section of medical student notes became more redundant over a patient's hospital course and as students gained clinical experience. These trends may be indicative of deficiencies in clinical knowledge or reasoning, as evidenced by performance on some standardized evaluations.

Health records as the basis of clinical coding: Is the quality adequate? A qualitative study of medical coders' perceptions

BACKGROUND

Health records are the basis of clinical coding. In Portugal, relevant diagnoses and procedures are abstracted and categorised using an internationally accepted classification system and the resulting codes, together with the administrative data, are then grouped into diagnosis-related groups (DRGs). Hospital reimbursement is partially calculated from the DRGs. Moreover, the administrative database generated with these data is widely used in research and epidemiology, among other purposes.

OBJECTIVE

To explore the perceptions of medical coders (medical doctors) regarding possible problems with health records that may affect the quality of coded data.

METHOD

A qualitative design using four focus groups sessions with 10 medical coders was undertaken between October and November 2017. The convenience sample was obtained from four public hospitals in Portugal. Questions related to problems with the coding process were developed from the literature and authors' expertise. The focus groups sessions were taped, transcribed and analysed to elicit themes.

RESULTS

There are several problems, identified by the focus groups, in health records that influence the coded data: the lack of or unclear documented information; the variability in diagnosis description; "copy & paste"; and the lack of solutions to solve these problems.

CONCLUSION AND IMPLICATIONS

The use of standards in health records, audits and physician awareness could increase the quality of health records, contributing to improvements in the quality of coded data, and in the fulfilment of its purposes (e.g. more accurate payments and more reliable research).

Unintended consequences of the electronic medical record on physicians in training and their mentors

For physicians in training and their mentors, the process of learning and teaching clinical medicine has become challenging in the electronic medical record (EMR) era. Trainees and their mentors exist in a milieu of incessant box checking and laborious documentation that has no clinical educational value, limits the time for teaching and curtails clinical cognitive skill development. These unintended consequences of the EMR are juxtaposed against the EMR's intended benefits of improved patient care and safety with reduced medical errors, improved clinical support systems, reduced potential for negligence with clinical data and metadata data supporting compliance with the standard of care. Although the mindset was technology would be the solution to many healthcare issues, there was not an appreciation of the cumulative impact of the non-educational workload on physician time and education. The EMR was intended to improve the efficiency of medical care and time management. It appears that the unintended consequences of the EMR with numerous checkboxes, automatic filling of computer screens, pre-worded templates, and automatic history and physical examination functions with detailed administrative oversight and compliance monitoring were not appreciated, and many believe that burden has overwhelmed the intended benefits of the EMR. This juxtaposition of the intended and unintended consequences of the EMR has left trainees and mentors struggling to optimise medical education and development of clinical skills while providing high-quality patient medical care. Physician educators must identify how to use the benefits of the EMR and overcome the unintended consequences. A major unintended consequence of the EMR is time dedicated to automate functions that detract from the time spent with mentors and patients. This time loss has the potential to restrict the physician from meeting the essential canons of medical informed consent and interfere with a physician meeting her fiduciary duties to the patient. To raise awareness and stimulate a search for solutions that benefit medical education and patient care, we will explore the intended and unintended consequences of the EMR and potential solutions using the intelligent systems of the EMR.

Electronic Health Records as an Educational Tool: Viewpoint

BACKGROUND

Electronic health records (EHRs) have been adopted by most hospitals and medical offices in the United States. Because of the rapidity of implementation, health care providers have not been able to leverage the full potential of the EHR for enhancing clinical care, learning, and teaching. Physicians are spending an average of 49% of their working hours on EHR documentation, chart review, and other indirect tasks related to patient care, which translates into less face time with patients.

OBJECTIVE

The purpose of this article is to provide a preliminary framework to guide the use of EHRs in teaching and evaluation of residents.

METHODS

First we discuss EHR educational capabilities that have not been reviewed in sufficient detail in the literature and expand our discussion for each educational activity with examples. We emphasize quality improvement of clinical notes as a basic foundational skill using a spreadsheet-based application as an assessment tool. Next, we integrate the six Accreditation Council for Graduate Medical Education (ACGME) Core Competencies and Milestones (CCMs) framework with the Reporter-Interpreter-Manager-Educator (RIME) model to expand our assessments of other areas of resident performance related to EHR use. Finally, we discuss how clinical utility, clinical outcome, and clinical reasoning skills can be assessed in the EHR.

RESULTS

We describe a pilot conceptual framework-CCM framework-to guide and demonstrate the use of the EHR for education in a clinical setting.

CONCLUSIONS

As EHRs and other supporting technologies evolve, medical educators should continue to look for new opportunities within the EHR for education. Our framework is flexible to allow adaptation and use in most training programs. Future research should assess the validity of such methods on trainees' education.

A Prescription for Note Bloat: An Effective Progress Note Template

BACKGROUND

United States hospitals have widely adopted electronic health records (EHRs). Despite the potential for EHRs to increase efficiency, there is concern that documentation quality has suffered.

OBJECTIVE

To examine the impact of an educational session bundled with a progress note template on note quality, length, and timeliness.

DESIGN

A multicenter, nonrandomized prospective trial.

SETTING

Four academic hospitals across the United States.

PARTICIPANTS

Intern physicians on inpatient internal medicine rotations at participating hospitals.

INTERVENTION

A task force delivered a lecture on current issues with documentation and suggested that interns use a newly designed best practice progress note template when writing daily progress notes.

MEASUREMENTS

Note quality was rated using a tool designed by the task force comprising a general impression score, the validated Physician Documentation Quality Instrument, 9-item version (PDQI-9), and a competency questionnaire. Reviewers documented number of lines per note and time signed.

RESULTS

Two hundred preintervention and 199 postintervention notes were collected. Seventy percent of postintervention notes used the template. Significant improvements were seen in the general impression score, all domains of the PDQI-9, and multiple competency items, including documentation of only relevant data, discussion of a discharge plan, and being concise while adequately complete. Notes had approximately 25% fewer lines and were signed on average 1.3 hours earlier in the day.

CONCLUSIONS

The bundled intervention for progress notes significantly improved the quality, decreased the length, and resulted in earlier note completion across 4 academic medical centers.

Sometimes determination and compromise thwart success: lessons learned from an effort to study copying and pasting in the electronic medical record

None

Perspectives and Uses of the Electronic Health Record Among US Pediatricians: A National Survey

Little is known about how existing electronic health records (EHRs) influence the practice of pediatric medicine. A total of 808 pediatricians participated in a survey about workflows using the EHR. The EHR was the most commonly used source of initial patient information. Seventy-two percent reported requiring between 2 and 10 minutes to complete an initial review of the EHR. Several moderately severe information barriers were reported regarding the display of information in the EHR. Pediatricians acquire information about new patients from EHRs more often than any other source. EHRs play a critical role in pediatric care but require improved design and efficiency.

Promoting Responsible Electronic Documentation: Validity Evidence for a Checklist to Assess Progress Notes in the Electronic Health Record

Construct: We aimed to develop an instrument to measure the quality of inpatient electronic health record- (EHR-) generated progress notes without requiring raters to review the detailed chart or know the patient.

BACKGROUND

Notes written in EHRs have generated criticism for being unnecessarily long and redundant, perpetuating inaccuracy and obscuring providers' clinical reasoning. Available assessment tools either focus on outpatient progress notes or require chart review by raters to develop familiarity with the patient.

APPROACH

We used medical literature, local expert review, and attending focus groups to develop and refine an instrument to evaluate inpatient progress notes. We measured interrater reliability and scored the selected-response elements of the checklist for a sample of 100 progress notes written by PGY-1 trainees on the general medicine service.

RESULTS

We developed an instrument with 18 selected-response items and four open-ended items to measure the quality of inpatient progress notes written in the EHR. The mean Cohen's kappa coefficient demonstrated good agreement at .67. The mean note score was 66.9% of maximum possible points (SD = 10.6, range = 34.4%-93.3%).

CONCLUSIONS

We present validity evidence in the domains of content, internal structure, and response process for a new checklist for rating inpatient progress notes. The scored checklist can be completed in approximately 7 minutes by a rater who is not familiar with the patient and can be done without extensive chart review. We further demonstrate that trainee notes show substantial room for improvement.

Characterizing the Source of Text in Electronic Health Record Progress Notes

This study analyzes inpatient progress notes to determine the documentation practices of medical students, residents, and hospitalists.

Detecting clinically relevant new information in clinical notes across specialties and settings

Background

Automated methods for identifying clinically relevant new versus redundant information in electronic health record (EHR) clinical notes is useful for clinicians and researchers involved in patient care and clinical research, respectively. We evaluated methods to automatically identify clinically relevant new information in clinical notes, and compared the quantity of redundant information across specialties and clinical settings.

Methods

Statistical language models augmented with semantic similarity measures were evaluated as a means to detect and quantify clinically relevant new and redundant information over longitudinal clinical notes for a given patient. A corpus of 591 progress notes over 40 inpatient admissions was annotated for new information longitudinally by physicians to generate a reference standard. Note redundancy between various specialties was evaluated on 71,021 outpatient notes and 64,695 inpatient notes from 500 solid organ transplant patients (April 2015 through August 2015).

Results

Our best method achieved at best performance of 0.87 recall, 0.62 precision, and 0.72 F-measure. Addition of semantic similarity metrics compared to baseline improved recall but otherwise resulted in similar performance. While outpatient and inpatient notes had relatively similar levels of high redundancy (61% and 68%, respectively), redundancy differed by author specialty with mean redundancy of 75%, 66%, 57%, and 55% observed in pediatric, internal medicine, psychiatry and surgical notes, respectively.

Conclusions

Automated techniques with statistical language models for detecting redundant versus clinically relevant new information in clinical notes do not improve with the addition of semantic similarity measures. While levels of redundancy seem relatively similar in the inpatient and ambulatory settings in the Fairview Health Services, clinical note redundancy appears to vary significantly with different medical specialties.

Resident Notes in an Electronic Health Record

Adoption of electronic health records (EHRs) has forced a transition in medical documentation, yet little is known about clinician documentation in the EHR. This study compares electronic inpatient progress notes written by residents pre- and post introduction of standardized note templates and investigates resident perceptions of EHR documentation. A total of 454 resident progress notes pre- and 610 notes post-template introduction were identified. Note length was 263 characters shorter ( P = .004) and mean end time was 73 minutes later ( P < .0001) with new template implementation. In subanalysis of 100 notes, the assessment and plan section was 46 words shorter with the new template ( P < .01). Among survey respondents, 89% liked the new note templates, 78% stated the new templates facilitated note completion. The resident focus group revealed ambivalence toward the EHR's contribution to note writing. Note templates resulted in shorter notes. Residents appreciate electronic note templates but are unsure if the EHR supports note writing overall.

The role of copy and paste function in orthopedic trauma progress notes

INTRODUCTION

The electronic medical record (EMR) is standard in institutions. While there is not concern for legibility of notes and access to charts, there is an ease of copy and paste for daily notes. This may not lead to accurate portrayal of patient's status. Our purpose was to evaluate the use of copy and paste functions in daily notes of patients with injuries at high risk for complications.

METHODS

IRB approval was obtained for a retrospective review. Inclusion criteria included patients aged 18 and older treated at our Level 1 Trauma Center after implementation of Epic Systems Corporation, Verona, WI, USA. Those who were surgically treated for bicondylar tibial plateau fracture, or open tibial shaft fracture type I or II were included. Manual comparison of daily progress to the previous day's note was carried out. Comparisons were made by evaluating the subjective, objective, and plan portions of the notes, coded nominally using 1 for a change 0 for remaining the same.

RESULTS

38 patients' charts were reviewed during a 10-month (July 2012-April 2013) period, and the average length of stay was 12 days (range: 2-35). A total of 418 notes were compared. The overall average of copied data was 85% daily. In the subjective portion, 85-97% of the data was copied on a daily basis and 71-92% of the data was copied within the objective portion of the notes. There were 15 medical complications necessitating intervention. Of these medical complications, the note the day after the complication reflected the event in 10 out of 15, or 70%, of the complications. Thus 5, or 30%, of the patients did not have notes reflecting the complication (p < 0.05). There were 7 complications related to the injuries: 4 cases of compartment syndrome, 1 case of foot drop, representing a change in neurologic status, an amputation, and a wound infection treated with antibiotics. Four of the 7 complications (57%) were not reflected in the notes the following day after the complication (p < 0.05). There were 54 planned returns to the operating room for procedures, yet 30 of the 54 (56%) notes regarding planned surgical procedures notes did not accurately report the plan for surgery (p < 0.05). There were 4 patients with unplanned trips to the operating room and 3 of the notes (75%) did not reflect this (p < 0.05). Twelve patients (32%) did not have notes accurately reflecting discharge plans and/or destination (p < 0.05).

DISCUSSION/CONCLUSION

Our results demonstrated widespread use of copy and paste function. We encourage evaluation of the charts by comparing notes to check and a plan to minimize this practice. There needs to be consistent note writing guidelines and appropriate templates used. This will decrease the inaccuracies in the chart and provide a clear picture of the patient, their injuries, and current status.

Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision

OBJECTIVE

The objective of this review is to summarize the state of the art of clinical decision support (CDS) circa 1990, review progress in the 25 year interval from that time, and provide a vision of what CDS might look like 25 years hence, or circa 2040.

METHOD

Informal review of the medical literature with iterative review and discussion among the authors to arrive at six axes (data, knowledge, inference, architecture and technology, implementation and integration, and users) to frame the review and discussion of selected barriers and facilitators to the effective use of CDS.

RESULT

In each of the six axes, significant progress has been made. Key advances in structuring and encoding standardized data with an increased availability of data, development of knowledge bases for CDS, and improvement of capabilities to share knowledge artifacts, explosion of methods analyzing and inferring from clinical data, evolution of information technologies and architectures to facilitate the broad application of CDS, improvement of methods to implement CDS and integrate CDS into the clinical workflow, and increasing sophistication of the end-user, all have played a role in improving the effective use of CDS in healthcare delivery.

CONCLUSION

CDS has evolved dramatically over the past 25 years and will likely evolve just as dramatically or more so over the next 25 years. Increasingly, the clinical encounter between a clinician and a patient will be supported by a wide variety of cognitive aides to support diagnosis, treatment, care-coordination, surveillance and prevention, and health maintenance or wellness.

The Effect of Adoption of an Electronic Health Record on Duplicate Testing

Background. The electronic health record (EHR) has been promoted as a tool to improve quality of patient care, reduce costs, and improve efficiency. There is little data to confirm that the use of EHR has reduced duplicate testing. We sought to evaluate the rate of performance of repeat transthoracic echocardiograms before and after the adoption of EHR. Methods. We retrospectively examined the rates of repeat echocardiograms performed before and after the implementation of an EHR system. Results. The baseline rate of repeat testing before EHR was 4.6% at six months and 7.6% at twelve months. In the first year following implementation of EHR, 6.6% of patients underwent a repeat study within 6 months, and 12.9% within twelve months. In the most recent year of EHR usage, 5.7% of patients underwent repeat echocardiography at six months and 11.9% within twelve months. All rates of duplicate testing were significantly higher than their respective pre-EHR rates (p < 0.01 for all). Conclusion. Our study failed to demonstrate a reduction in the rate of duplicate echocardiography testing after the implementation of an EHR system. We feel that this data, combined with other recent analyses, should promote a more rigorous assessment of the initial claims of the benefits associated with EHR implementation.

Health information technology needs help from primary care researchers

While health information technology (HIT) efforts are beginning to yield measurable clinical benefits, more is needed to meet the needs of patients and clinicians. Primary care researchers are uniquely positioned to inform the evidence-based design and use of technology. Research strategies to ensure success include engaging patient and clinician stakeholders, working with existing practice-based research networks, and using established methods from other fields such as human factors engineering and implementation science. Policies are needed to help support primary care researchers in evaluating and implementing HIT into everyday practice, including expanded research funding, strengthened partnerships with vendors, open access to information systems, and support for the Primary Care Extension Program. Through these efforts, the goal of improved outcomes through HIT can be achieved.

Electronic health record innovations for healthier patients and happier doctors

This special issue explores a range of health information technology (HIT) issues that can help primary care practices and patients. Findings address the design of HIT systems, primarily electronic health records (EHRs), the utility of various functionalities, and implementation strategies that ensure the greatest value. The articles also remind us that, while HIT can support the delivery of care, it is not a panacea. To be effective, functionality needs to be relevant and timely for both the clinician and patient. Prompts and better documentation can improve care, and "prompt fatigue" is not inevitable. Information presented within EHRs needs to be actionable. There is an ongoing tension between information overload and the right-and helpful-information. Even the order of presentation of information can make a difference in the outcome. Whether supported by HIT or not, basic tenants of care, such as including the whole care team in trainings, communicating with other providers, and engaging patients, remain essential. The studies in this issue will prove useful for informatics developers, practices and health systems making HIT decisions, and care teams refining HIT to support the needs of their patients.

Automated methods for the summarization of electronic health records

Objectives This review examines work on automated summarization of electronic health record (EHR) data and in particular, individual patient record summarization. We organize the published research and highlight methodological challenges in the area of EHR summarization implementation.

Target audience The target audience for this review includes researchers, designers, and informaticians who are concerned about the problem of information overload in the clinical setting as well as both users and developers of clinical summarization systems.

Scope Automated summarization has been a long-studied subject in the fields of natural language processing and human–computer interaction, but the translation of summarization and visualization methods to the complexity of the clinical workflow is slow moving. We assess work in aggregating and visualizing patient information with a particular focus on methods for detecting and removing redundancy, describing temporality, determining salience, accounting for missing data, and taking advantage of encoded clinical knowledge. We identify and discuss open challenges critical to the implementation and use of robust EHR summarization systems.

Clinical documentation in the 21st century: executive summary of a policy position paper from the American College of Physicians

Clinical documentation was developed to track a patient's condition and communicate the author's actions and thoughts to other members of the care team. Over time, other stakeholders have placed additional requirements on the clinical documentation process for purposes other than direct care of the patient. More recently, new information technologies, such as electronic health record (EHR) systems, have led to further changes in the clinical documentation process. Although computers and EHRs can facilitate and even improve clinical documentation, their use can also add complexities; new challenges; and, in the eyes of some, an increase in inappropriate or even fraudulent documentation. At the same time, many physicians and other health care professionals have argued that the quality of the systems being used for clinical documentation is inadequate. The Medical Informatics Committee of the American College of Physicians has undertaken this review of clinical documentation in an effort to clarify the broad range of complex and interrelated issues surrounding clinical documentation and to suggest a path forward such that care and clinical documentation in the 21st century best serve the needs of patients and families.

The effectiveness of a bundled intervention to improve resident progress notes in an electronic health record

Providers nationally have observed a decline in the quality of documentation after implementing electronic health records (EHRs). In this pilot study, we examined the effectiveness of an intervention bundle designed to improve resident progress notes written in an EHR and to establish the reliability of an audit tool used to evaluate notes. The bundle consisted of establishing note-writing guidelines, developing an aligned note template, and educating interns about the guidelines and using the template. Twenty-five progress notes written by pediatric interns before and after this intervention were examined using an audit tool. Reliability of the tool was evaluated using the intraclass correlation coefficient (ICC). The total score of the audit tool was summarized in terms of means and standard deviation. Individual item responses were summarized using percentages and compared between the pre- and postintervention assessment using the Fisher exact test. The ICC for the audit tool was 0.96 (95% confidence interval: 0.91-0.98). A significant improvement in the total note score and in questions related to note clutter was seen. No significant improvement was seen for questions related to copy-paste. The study suggests that an intervention bundle can lead to some improvements in note writing.