QNOTE: an instrument for measuring the quality of EHR clinical notes

Your note, your way: how to write an inpatient progress note accurately and efficiently as an intern

A physician's progress note is an essential piece of documentation regarding key events and the daily status of patients during their hospital stay. It serves not only as a communication tool between care team members, but also chronicles clinical status and pertinent updates to their medical care. Despite the importance of these documents, little literature exists on how to help residents to improve the quality of their daily progress notes. A narrative literature review of English language literature was performed and summated to provide recommendations on how to write an inpatient progress note more accurately and efficiently. In addition, the authors will also introduce a method to build a personal template with the goal of extracting relevant data automatically to reduce clicks for an inpatient progress note in the electronic medical record system.

Assessing the Assessment-Developing and Deploying a Novel Tool for Evaluating Clinical Notes' Diagnostic Assessment Quality

BACKGROUND

Ambulatory diagnostic errors are increasingly being recognized as an important quality and safety issue, and while measures of diagnostic quality have been sought, tools to evaluate diagnostic assessments in the medical record are lacking.

OBJECTIVE

To develop and test a tool to measure diagnostic assessment note quality in primary care urgent encounters and identify common elements and areas for improvement in diagnostic assessment.

DESIGN

Retrospective chart review of urgent care encounters at an urban academic setting.

PARTICIPANTS

Primary care physicians.

MAIN MEASURES

The Assessing the Assessment (ATA) instrument was evaluated for inter-rater reliability, internal consistency, and findings from its application to EHR notes.

KEY RESULTS

ATA had reasonable performance characteristics (kappa 0.63, overall Cronbach's alpha 0.76). Variability in diagnostic assessment was seen in several domains. Two components of situational awareness tended to be well-documented ("Don't miss diagnoses" present in 84% of charts, red flag symptoms in 87%), while Psychosocial context was present only 18% of the time.

CONCLUSIONS

The ATA tool is a promising framework for assessing and identifying areas for improvement in diagnostic assessments documented in clinical encounters.

Comparison of clinical note quality between an automated digital intake tool and the standard note in the emergency department

BACKGROUND

Medical encounters require an efficient and focused history of present illness (HPI) to create differential diagnoses and guide diagnostic testing and treatment. Our aim was to compare the HPI of notes created by an automated digital intake tool versus standard medical notes created by clinicians.

METHODS

Prospective trial in a quaternary academic Emergency Department (ED). Notes were compared using the 5-point Physician Documentation Quality Instrument (PDQI-9) scale and the Centers for Medicare & Medicaid Services (CMS) level of complexity index. Reviewers were board certified emergency medicine physicians blinded to note origin. Reviewers received training and calibration prior to note assessments. A difference of 1 point was considered clinically significant. Analysis included McNemar's (binary), Wilcoxon-rank (Likert), and agreement with Cohen's Kappa.

RESULTS

A total of 148 ED medical encounters were charted by both digital note and standard clinical note. The ability to capture patient information was assessed through comparison of note content across paired charts (digital-standard note on the same patient), as well as scores given by the reviewers. Reviewer agreement was kappa 0.56 (CI 0.49-0.64), indicating moderate level of agreement between reviewers scoring the same patient chart. Considering all 18 questions across PDQI-9 and CMS scales, the average agreement between standard clinical note and digital note was 54.3% (IQR 44.4-66.7%). There was a moderate level of agreement between content of standard and digital notes (kappa 0.54, 95%CI 0.49-0.60). The quality of the digital note was within the 1 point clinically significant difference for all of the attributes, except for conciseness. Digital notes had a higher frequency of CMS severity elements identified.

CONCLUSION

Digitally generated clinical notes had moderate agreement compared to standard clinical notes and within the one point clinically significant difference except for the conciseness attribute. Digital notes more reliably documented billing components of severity. The use of automated notes should be further explored to evaluate its utility in facilitating documentation of patient encounters.

Writing Is Thinking: Implementation and Evaluation of an Internal Medicine Residency Clinical Reasoning and Documentation Curriculum

With increasingly complicated patients and faster throughput, time for thorough critical thinking and thoughtful clinical documentation is limited, especially in the training environment. Advocating for the value of clinical documentation as a robust opportunity for critical thinking, we describe the implementation and evaluation of a clinical reasoning and documentation curriculum for internal medicine residents. Our curriculum employed facilitated discussion, practical application, and a resident-as-teacher model. Resident surveys showed improved perceptions of the clinical and educational value of clinical documentation. Residents reported increased feedback to interns about their documentation and more appreciation of documentation as a venue for critical thinking.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-022-01570-5.

The Analyzation of Change in Documentation due to the Introduction of Electronic Patient Records in Hospitals-A Systematic Review

The major impact on healthcare through the ongoing digital transformation and new technologies results in opportunities for improving quality of care. Electronic patient records (EPR) are a substantial part in this transformation, even though their influence on documentation remains often unclear. This review aims to answer the question of which effect the introduction of the EPR has on the documentation proper in hospitals. To do this, studies are reviewed that analyze the documentation itself, rather than merely conducting interviews or surveys about it. Several databases were searched in this systematic review (PubMed including PubMed, PubMed Central and Medline; PDQ Evidence; Web of Science Core Collection; CINHAL). To be included, studies needed to analyze written documentation and empirical data, be in either German or English language, published between 2010 and 2020, conducted in a hospital setting, focused on transition from paper-based to electronic patient records, and peer reviewed. Quantitative, qualitative and mixed methods studies were included. Studies were independently screened for inclusion by two researchers in three stages (title, abstract, full text) and, in case of disagreement, discussed with a third person from the research team until consensus was reached. The main outcome assessed was whether the studies indicated a negative or positive effect on documentation (e.g. changing the completeness of documentation) by introducing an EPR. Mixed Methods Appraisal Tool was used to assess the individual risk of bias in the included studies. Overall, 264 studies were found. Of these, 17 met the inclusion criteria and were included in this review. Of all included studies, 11 of 17 proved a positive effect of the introduction of the EPR on documentation such as an improved completeness or guideline adherence of the documentation. Six of 17 showed a mixed effect with positive and negative or no changes. No study showed an exclusively negative effect. Most studies found a positive effect of EPR introduction on documentation. However, it is difficult to draw specific conclusions about how the EPR affects or does not affect documentation since the included studies examined a variety of outcomes. As a result, various scenarios are conceivable with higher or reduced burden for practitioners. Additionally, the impact on treatment remains unclear.

Development and Validation of a Machine Learning Model for Automated Assessment of Resident Clinical Reasoning Documentation

BACKGROUND

Residents receive infrequent feedback on their clinical reasoning (CR) documentation. While machine learning (ML) and natural language processing (NLP) have been used to assess CR documentation in standardized cases, no studies have described similar use in the clinical environment.

OBJECTIVE

The authors developed and validated using Kane's framework a ML model for automated assessment of CR documentation quality in residents' admission notes.

DESIGN, PARTICIPANTS, MAIN MEASURES

Internal medicine residents' and subspecialty fellows' admission notes at one medical center from July 2014 to March 2020 were extracted from the electronic health record. Using a validated CR documentation rubric, the authors rated 414 notes for the ML development dataset. Notes were truncated to isolate the relevant portion; an NLP software (cTAKES) extracted disease/disorder named entities and human review generated CR terms. The final model had three input variables and classified notes as demonstrating low- or high-quality CR documentation. The ML model was applied to a retrospective dataset (9591 notes) for human validation and data analysis. Reliability between human and ML ratings was assessed on 205 of these notes with Cohen's kappa. CR documentation quality by post-graduate year (PGY) was evaluated by the Mantel-Haenszel test of trend.

KEY RESULTS

The top-performing logistic regression model had an area under the receiver operating characteristic curve of 0.88, a positive predictive value of 0.68, and an accuracy of 0.79. Cohen's kappa was 0.67. Of the 9591 notes, 31.1% demonstrated high-quality CR documentation; quality increased from 27.0% (PGY1) to 31.0% (PGY2) to 39.0% (PGY3) (p < .001 for trend). Validity evidence was collected in each domain of Kane's framework (scoring, generalization, extrapolation, and implications).

CONCLUSIONS

The authors developed and validated a high-performing ML model that classifies CR documentation quality in resident admission notes in the clinical environment-a novel application of ML and NLP with many potential use cases.

The Impact of Structured and Standardized Documentation on Documentation Quality; a Multicenter, Retrospective Study

The reuse of healthcare data for various purposes will become increasingly important in the future. To enable the reuse of clinical data, structured and standardized documentation is conditional. However, the primary purpose of clinical documentation is to support high-quality patient care. Therefore, this study investigated the effect of increased structured and standardized documentation on the quality of notes in the Electronic Health Record. A multicenter, retrospective design was used to assess the difference in note quality between 144 unstructured and 144 structured notes. Independent reviewers measured note quality by scoring the notes with the Qnote instrument. This instrument rates all note elements independently using and results in a grand mean score on a 0–100 scale. The mean quality score for unstructured notes was 64.35 (95% CI 61.30–67.35). Structured and standardized documentation improved the Qnote quality score to 77.2 (95% CI 74.18–80.21), a 12.8 point difference (p < 0.001). Furthermore, results showed that structured notes were significantly longer than unstructured notes. Nevertheless, structured notes were more clear and concise. Structured documentation led to a significant increase in note quality. Moreover, considering the benefits of structured data recording in terms of data reuse, implementing structured and standardized documentation into the EHR is recommended.

Improving Medical Student Inpatient Documentation Through Feedback Using a Note Assessment Tool

Introduction

Documentation within the Electronic Health Record (EHR) is an essential skill for medical students to succeed in residency and post-residency training. The increased use of medical student progress notes for billable services raises the need for the education and assessment of quality note writing. We hypothesized that structured note feedback using a note assessment tool would improve the quality of medical student inpatient progress notes.

Methods

We conducted a retrospective study to review the quality of student inpatient progress notes written before and after structured feedback using the Responsible Electronic Documentation (RED) checklist throughout a third-year internal medicine clerkship. The first intervention group received feedback from clerkship directors in the 2017-2018 academic year and the second intervention group received feedback from ward residents/attendings in the 2018-2019 academic year. Within each intervention group, the total note scores from pre and post-intervention were compared.

Results

Feedback from clerkship directors yielded a greater increase in students’ total note score from pre to post-intervention compared to ward resident/attending feedback (F(1,255) = 12.84, p < 0.001). Cohen’s d effect size value was greater for the clerkship director feedback arm (d=0.71) compared to the ward resident/attending feedback arm (d=0.24). Post-hoc analyses using dependent sample t-tests revealed that there were significant increases in total note scores from pre to post-intervention for both the clerkship director arm (t(123) = 8.26, p < 0.001, d = 0.71) and the ward resident/attending arm (t(132) = 2.85, p = 0.005, d = 0.24).

Conclusion

Clerkship director feedback led to a greater increase in medical student documentation compared to ward attending/resident feedback. Nonetheless, structured feedback with a note assessment tool, whether from clerkship directors or ward attendings/residents, leads to a significant improvement in medical student documentation. Though there are various methods for providing feedback, educators can use the RED checklist to provide clear guidelines that will facilitate note-writing feedback.

A Practical Approach for Monitoring the Use of Copy-Paste in Clinical Notes

The use of copy-paste in authoring clinical notes has been widely embraced by busy providers, but inappropriate copy-paste has been lambasted by critics for introducing risks related to patient safety and regulatory compliance. At an integrated academic health system with over 4,100 providers writing notes, we developed a pragmatic approach to assess the use of copy-paste. From January 1-December 31, 2020, approximately 2.3M inpatient notes and 6.6M ambulatory clinic notes were authored in our electronic health record. Of the inpatient notes, 42% used copy-paste, and 19% of overall note content was copied; in ambulatory notes, 18% used copy-paste and 12% of note content was copied. We describe an approach for including providers' copy-paste usage statistics into the ongoing professional practice evaluation process required for hospital accreditation, thereby offering individual training opportunities related to the lack of use of copy-paste or its potential overuse.

Development of a Clinical Reasoning Documentation Assessment Tool for Resident and Fellow Admission Notes: a Shared Mental Model for Feedback

BACKGROUND

Residents and fellows receive little feedback on their clinical reasoning documentation. Barriers include lack of a shared mental model and variability in the reliability and validity of existing assessment tools. Of the existing tools, the IDEA assessment tool includes a robust assessment of clinical reasoning documentation focusing on four elements (interpretive summary, differential diagnosis, explanation of reasoning for lead and alternative diagnoses) but lacks descriptive anchors threatening its reliability.

OBJECTIVE

Our goal was to develop a valid and reliable assessment tool for clinical reasoning documentation building off the IDEA assessment tool.

DESIGN, PARTICIPANTS, AND MAIN MEASURES

The Revised-IDEA assessment tool was developed by four clinician educators through iterative review of admission notes written by medicine residents and fellows and subsequently piloted with additional faculty to ensure response process validity. A random sample of 252 notes from July 2014 to June 2017 written by 30 trainees across several chief complaints was rated. Three raters rated 20% of the notes to demonstrate internal structure validity. A quality cut-off score was determined using Hofstee standard setting.

KEY RESULTS

The Revised-IDEA assessment tool includes the same four domains as the IDEA assessment tool with more detailed descriptive prompts, new Likert scale anchors, and a score range of 0-10. Intraclass correlation was high for the notes rated by three raters, 0.84 (95% CI 0.74-0.90). Scores ≥6 were determined to demonstrate high-quality clinical reasoning documentation. Only 53% of notes (134/252) were high-quality.

CONCLUSIONS

The Revised-IDEA assessment tool is reliable and easy to use for feedback on clinical reasoning documentation in resident and fellow admission notes with descriptive anchors that facilitate a shared mental model for feedback.

Improving trainee clinical documentation through a novel curriculum in internal medicine

BACKGROUND

Clinical documentation is a key component of practice. Trainees rarely receive formal training in documentation or assessment of their documentation. Effective methods of improving documentation remain unknown.

OBJECTIVE

The objective of this study was to determine if the implementation of a documentation curriculum led to improvement in admission note quality.

DESIGNS

Admission notes written prior to implementation of the curriculum and after the curriculum intervention were assessed. Notes were assessed from two-time frames for both years to account for improvement with time not associated with the intervention.

SETTINGS AND PARTICIPANTS

Admission notes written by University of Cincinnati interns were assessed.

INTERVENTIONS

The documentation curriculum consisted of educational sessions and routine admission note assessments with feedback.

MAIN OUTCOMES AND MEASURES

Admission notes were assessed via the 16 checklist items and two global assessment items of the Admission Note Assessment Tool (ANAT).

RESULTS

Six ANAT items showed statistically significant differences. The review of systems item improved with the intervention only (odds ratio: 3.61, p < .001) while the assessment and plan item 1 and global assessment item 2 improved with time only (β = .08, p = .03 and β = .25, p = .02, respectively) in univariate models. In univariate models the physical exam item, diagnostic data item 2, and global assessment item 1 showed improvement with both intervention and time, respectively, with additive effects seen in models with both intervention and time.

CONCLUSION

Several aspects of documentation can improve with a formal documentation curriculum which includes a routine assessment with feedback, and some aspects of documentation improve with time.

Impact of a medical scribe on clinical efficiency and quality in an academic general internal medicine practice

Background

Scribes have been proposed as an intervention to decrease physician electronic health record (EHR) workload and improve clinical quality. We aimed to assess the impact of a scribe on clinical efficiency and quality in an academic internal medicine practice.

Methods

Six faculty physicians worked with one scribe at an urban academic general internal medicine clinic April through June 2017. Patient visits during the 3 months prior to intervention (baseline, n = 789), unscribed visits during the intervention (concurrent control, n = 605), and scribed visits (n = 579) were included in the study. Clinical efficiency outcomes included time to close encounter, patient time in clinic, and number of visits per clinic session. Quality outcomes included EHR note quality, rates of medication and immunization review, population of patient instructions, reconciliation of outside information, and completion of preventative health recommendations.

Results

Median time to close encounter (IQR) was lower for scribed visits [0.4 (4.8) days] compared to baseline and unscribed visits [1.2 (5.9) and 2.9 (5.4) days, both p < 0.001]. Scribed notes were more likely to have a clear history of present illness (HPI) [OR = 7.30 (2.35–22.7), p = 0.001] and sufficient HPI information [OR = 2.21 (1.13–4.35), p = 0.02] compared to unscribed notes. Physicians were more likely to review the medication list during scribed vs. baseline visits [OR = 1.70 (1.22–2.35), p = 0.002]. No differences were found in the number of visits per clinic session, patient time in clinic, completion of preventative health recommendations, or other outcomes.

Conclusions

Working with a scribe in an academic internal medicine practice was associated with more timely documentation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06710-y.

Effect of Outpatient Note Templates on Note Quality: NOTE (Notation Optimization through Template Engineering) Randomized Clinical Trial

BACKGROUND

This is the first randomized controlled trial evaluating the impact of note template design on note quality using a simulated patient encounter and a validated assessment tool.

OBJECTIVE

To compare note quality between two different templates using a novel randomized clinical simulation process.

DESIGN

A randomized non-blinded controlled trial of a standard note template versus redesigned template.

PARTICIPANTS

PGY 1-3 IM residents.

INTERVENTIONS

Residents documented the simulated patient encounter using one of two templates. The standard template was modeled after the usual outpatient progress note. The new template placed the assessment and plan section in the beginning, grouped subjective data into the assessment, and deemphasized less useful elements.

MAIN MEASURES

Note length; time to note completion; note template evaluation by resident authors; note evaluation by faculty reviewers.

KEY RESULTS

36 residents participated, 19 randomized to standard template, 17 to new. New template generated shorter notes (103 vs 285 lines, p < 0.001) that took the same time to complete (19.8 vs 21.6 min, p = 0.654). Using a 5-point Likert scale, residents considered new notes to have increased visual appeal (4 vs 3, p = 0.05) and less redundancy and clutter (4 vs 3, p = 0.006). Overall template satisfaction was not statistically different. Faculty reviewers rated the standard note more up-to-date (4.3 vs 2.7, p = 0.001), accurate (3.9 vs 2.6, p = 0.003), and useful (4 vs 2.8, p = 0.002), but less organized (3.3 vs 4.5, p < 0.001). Total quality was not statistically different.

CONCLUSIONS

Residents rated the new note template more visually appealing, shorter, and less cluttered. Faculty reviewers rated both note types equivalent in the overall quality but rated new notes inferior in terms of accuracy and usefulness though better organized. This study demonstrates a novel method of a simulated clinical encounter to evaluate note templates before the introduction into practice.

TRIAL REGISTRATION

ClinicalTrials.gov ID: NCT04333238.

Improving clinical documentation: introduction of electronic health records in paediatrics

Medical records are crucial facet of a patient’s journey. These provide the clinician with a permanent record of the patient’s illness and ongoing medical care, thus enabling informed clinical decisions. In many hospitals, patient medical records are written on paper. However, written notes are liable to misinterpretation due to illegibility and misplacement. This can affect the patient’s medical care and has medico-legal implications. Electronic patient records (EPR) have been gradually introduced to replace patient’s paper notes with the aim of providing a more reliable record-keeping system. It is perceived that EPR improve the quality and efficiency of patient care. The paediatric department at Queen’s Hospital Burton uses a mix of paper notes and computerised medical records. Clinicians primarily use paper notes for admission clerking, ward rounds, ward reviews and outpatient clinic consultations. Laboratory tests, imaging results and prescription requests are executed via the EPR system. Documentation by nurses is also carried out electronically. We aimed to improve and standardise clinical documentation of paediatric admissions and ward round notes by developing electronic proforma for initial paediatric clerking, ward rounds and patient reviews. This quality improvement project improved clinical documentation on the paediatric wards and enhanced patient record-keeping, boosted clinical information-sharing and streamlined patient journey. It fulfilled various generic multidisciplinary record keeping audit tool standards endorsed by the Royal College of Physicians by 100%. We undertook a staff survey to investigate the opinion before and after implementing the electronic health record. Doctors, nurses and healthcare support workers overwhelmingly supported the quality, usefulness, completeness of specified fields and practicality of the electronic records.

Evaluation of interventions to improve inpatient hospital documentation within electronic health records: a systematic review

OBJECTIVE

Despite the widespread and increasing use of electronic health records (EHRs), the quality of EHRs is problematic. Efforts have been made to address reasons for poor EHR documentation quality. Previous systematic reviews have assessed intervention effectiveness within the outpatient setting or paper documentation. The purpose of this systematic review was to assess the effectiveness of interventions seeking to improve EHR documentation within an inpatient setting.

MATERIALS AND METHODS

A search strategy was developed based on elaborated inclusion/exclusion criteria. Four databases, gray literature, and reference lists were searched. A REDCap data capture form was used for data extraction, and study quality was assessed using a customized tool. Data were analyzed and synthesized in a narrative, semiquantitative manner.

RESULTS

Twenty-four studies were included in this systematic review. Owing to high heterogeneity, quantitative comparison was not possible. However, statistically significant results in interventions and affected outcomes were analyzed and discussed. Education and implementation of a new EHR reporting system were the most successful interventions, as evidenced by significantly improved EHR documentation.

DISCUSSION

Heterogeneity of interventions, outcomes, document type, EHR user, and other variables led to difficulty in measuring EHR documentation quality and effectiveness of interventions. However, the use of education as a primary intervention aligned closely with existing literature in similar fields.

CONCLUSIONS

Interventions implemented to enhance EHR documentation are highly variable and require standardization. Emphasis should be placed on this novel area of research to improve communication between healthcare providers and facilitate data sharing between centers and countries.

PROSPERO Registration Number: CRD42017083494.

Development and Optimization of Clinical Informatics Infrastructure to Support Bioinformatics at an Oncology Center

Translational bioinformatics for therapeutic discovery requires the infrastructure of clinical informatics. In this chapter, we describe the clinical informatics components needed for successful implementation of translational research at a cancer center. This chapter is meant to be an introduction to those clinical informatics concepts that are needed for translational research. For a detailed account of clinical informatics, the authors will guide the reader to comprehensive resources. We provide examples of workflows from Moffitt Cancer Center led by Drs. Perkins and Markowitz. This perspective represents an interesting collaboration as Dr. Perkins is the Chief Medical Information Officer and Dr. Markowitz is a translational researcher in Melanoma with an active informatics component to his laboratory to study the mechanisms of resistance to checkpoint blockade and an active member of the clinical informatics team.

Evaluation of an advanced practice provider emergency department critical care step-down unit

OBJECTIVE

In response to concerns about patient care and safety, our urban, tertiary care, Level 1 trauma center adult emergency department (ED) created an advanced practice provider-staffed critical care step-down unit (CCSU). We conducted a comprehensive evaluation of the CCSU's impact on patient care, safety, and ED operations.

METHODS

We compared ED length of stay, return visits to the ED within 72 hours, billing code assignments (current procedural terminology evaluation and management [CPT E&M] codes), and quality of electronic health record documentation per QNOTE for the 2 years after the CCSU was initiated (CCSU period) versus before its initiation (pre-CCSU period).

RESULTS

There were 31,418 critical care ED patient visits in the pre-CCSU period and 33,396 in the CCSU period. Median ED length of stay did not change overall between the CCSU versus pre-CCSU period (∆1 [95% confidence interval (CI) = -2.4, 4.4] minutes), but decreased for patients who remained in the critical care suites (∆-4 [95% CI = -7.8, -0.2] minutes). 72-hour return ED visits also did not change overall (∆0% [95% CI = -0.1, 0]), but decreased for patients who remained in the critical care suites (∆0.4% [95% CI = -0.05, -0.4]). CPT E&M billing increased for highest-level visits (99,291: ∆1.3% [95% CI= 0.5, 2.0]). Quality of electronic health record documentation as measured by QNOTE also improved (∆11.5% [95% CI = 4.9, 18.1]).

CONCLUSION

This ED's CCSU performance metrics indicate at least moderate improvement in ED length of stay, 72-hour return visits, critical care patient billing, and electronic health record documentation. EDs elsewhere can consider implementation of this advanced practice provider-staffed solution to improvement in critical care in ED.

Improved efficiency of patient admission with electronic health records in neurosurgery

BACKGROUND

Electronic health records (EHRs) may be controversial but they have the potential to improve patient care. We investigated whether the introduction of an electronic template-based admission form for the collection of information about the patient's medical history and neurological and clinical state at admission in the neurosurgical unit might have an impact on the quality of documentation in a discharge record and the amount of time taken to produce this documentation.

METHOD

A new digital template-based admission form (EHR) was developed and assessed with QNOTE, an assessment tool of medical notes with standardised criteria and the possibility to benchmark the quality of documentations. This was compared to 30 prior paper-based handwritten documentations (HWD) regarding the utilisation of these medical notes for dictation of medical discharge records.

RESULTS

Implementation of the EHR significantly improved the quality of patient admission documentation with a QNOTE mean grand score of 87 ± 22 (p < 0.0001) compared to prior HWD with 44 ± 30. The mean documentation time for HWD was 8.1 min ± 4.1 min and the dictation time for discharge records was 10.6 min ± 3.5 min. After implementation of EHR, the documentation time increased slightly to 9.6 min ± 2.3 min (n.s.), while the time for dictation of discharge records was reduced to 5.1 min ± 1.2 min (p < 0.0001). There was a clear correlation between a higher quality of documentation and a higher needed documentation time as well as higher quality of documentation and lower dictation times of discharge records.

CONCLUSION

Implementation of the EHR improved the quality of patient admission documentation and reduced the dictation time of discharge records.

IMPLICATIONS

It is crucial to involve stakeholders and users of EHRs in a timely manner during the stage of development and implementation phase to ensure optimal results and better usability.

Physician use of speech recognition versus typing in clinical documentation: A controlled observational study

IMPORTANCE

Speech recognition (SR) is increasingly used directly by clinicians for electronic health record (EHR) documentation. Its usability and effect on quality and efficiency versus other documentation methods remain unclear.

OBJECTIVE

To study usability and quality of documentation with SR versus typing.

DESIGN

In this controlled observational study, each subject participated in two of five simulated outpatient scenarios. Sessions were recorded with Morae® usability software. Two notes were documented into the EHR per encounter (one dictated, one typed) in randomized order. Participants were interviewed about each method's perceived advantages and disadvantages. Demographics and documentation habits were collected via survey. Data collection occurred between January 8 and February 8, 2019, and data analysis was conducted from February through September of 2019.

SETTING

Brigham and Women's Hospital, Boston, Massachusetts, USA.

PARTICIPANTS

Ten physicians who had used SR for at least six months.

MAIN OUTCOMES AND MEASURES

Documentation time, word count, vocabulary size, number of errors, number of corrections and quality (clarity, completeness, concision, information sufficiency and prioritization).

RESULTS

Dictated notes were longer than typed notes (320.6 vs. 180.8 words; p = 0.004) with more unique words (170.9 vs. 120.4; p = 0.01). Documentation time was similar between methods, with dictated notes taking slightly less time to complete than typed notes. Typed notes had more uncorrected errors per note than dictated notes (2.9 vs. 1.5), although most were minor misspellings. Dictated notes had a higher mean quality score (7.7 vs. 6.6; p = 0.04), were more complete and included more sufficient information.

CONCLUSIONS AND RELEVANCE

Participants felt that SR saves them time, increases their efficiency and allows them to quickly document more relevant details. Quality analysis supports the perception that SR allows for more detailed notes, but whether dictation is objectively faster than typing remains unclear, and participants described some scenarios where typing is still preferred. Dictation can be effective for creating comprehensive documentation, especially when physicians like and feel comfortable using SR. Research is needed to further improve integration of SR with EHR systems and assess its impact on clinical practice, workflows, provider and patient experience, and costs.

Development and Establishment of Initial Validity Evidence for a Novel Tool for Assessing Trainee Admission Notes

BACKGROUND

Documentation is a key component of practice, yet few curricula have been published to teach trainees proper note construction. Additionally, a gold standard for assessing note quality does not exist, and no documentation assessment tools integrate with established competency-based frameworks.

OBJECTIVE

To develop and establish initial validity evidence for a novel tool that assesses key components of trainee admission notes and maps to the Accreditation Council for Graduate Medical Education (ACGME) milestone framework.

DESIGN

Using an iterative, consensus building process we developed the Admission Note Assessment Tool (ANAT). Pilot testing was performed with both the supervising attending and study team raters not involved in care of the patients. The finalized tool was piloted with attendings from other institutions.

PARTICIPANTS

Local experts participated in tool development and pilot testing. Additional attending physicians participated in pilot testing.

MAIN MEASURES

Content, response process, and internal structure validity evidence was gathered using Messick's framework. Inter-rater reliability was assessed using percent agreement.

KEY RESULTS

The final tool consists of 16 checklist items and two global assessment items. Pilot testing demonstrated rater agreement of 72% to 100% for checklist items and 63% to 70% for global assessment items. Note assessment required an average of 12.3 min (SD 3.7). The study generated validity evidence in the domains of content, response process, and internal structure for use of the tool in rating admission notes.

CONCLUSIONS

The ANAT assesses individual components of a note, incorporates billing criteria, targets note "bloat," allows for narrative feedback, and provides global assessments mapped to the ACGME milestone framework. The ANAT can be used to assess admission notes by any attending and at any time after note completion with minimal rater training. The ANAT allows programs to implement routine note assessment for multiple functions with the use of a single tool.

Pilot trial of semi-automated medical note writing using lexeme hypotheses

Clinicians write a billion free text notes per year. These notes are typically replete with errors of all types. No established automated method can extract data from this treasure trove. The practice of medicine therefore remains haphazard and chaotic, resulting in vast economic waste. The lexeme hypotheses are based on our analysis of how records are created. They enable a computer system to predict what issue a clinician will need to address next, based on the environment in which the clinician is working, and what responses the clinician has selected to date. The system uses a lexicon storing the issues (queries) and a range of responses to the issues. When the clinician selects a response, a text fragment is added to the output file. In the first phase of this work, the notes of 69 returning hemophilia patients were scrutinized, and the lexicon was expanded to 847 lexeme queries and 7995 responses to enable the construction of completed notes. The quality of lexeme-generated notes from 20 consecutive subjects was then compared to the clinicians' conventional clinic notes. The system generated grammatically correct notes. In comparison to the traditional clinic note, the lexeme-generated notes were more complete (88 % compared with 62 %), and had less typographical and grammatical errors (0.8 versus 3.5 errors per note). The system notes and traditional notes averaged about 800 words, but the traditional notes had a much wider distribution of lengths. The note-creation rate from marshalling the data to completion using the system averaged 80 wpm, twice as fast as the typical clinician can type. The lexeme method generates more complete, grammatical and organized notes faster than traditional methods. The notes are completely computerized at inception, and they incorporate prompts for clinicians to address otherwise overlooked items. This pilot justifies further exploration of this methodology.

Implementing a clinical-educator curriculum to enrich internal medicine residents' teaching capacity

INTRODUCTION

Physicians-in-training (residents) are typically the primary educators for medical students during clinical clerkships. However, residents are not formally trained to teach or to assess their teaching. The aim of this study was to assess the implementation of a clinical educator rotation aimed at developing residents' competencies related to clinical teaching.

METHODS

A mixed-methods approach was used to develop and assess the clinical educator rotation at a teaching community hospital. Internal medicine residents who participated in the rotation and consented to the research were assigned to the clinical educator trainee (CET) group, the remaining residents were assigned to the control group. Osteopathic medical students rotating in the medicine service line were invited to participate. The study used descriptive and qualitative analyses to measure primary and secondary outcomes.

RESULTS

The primary outcome measure showed a positive change in resident knowledge, skills and behaviors in communication, reflection, feedback, precepting, and facilitation. Medical student perceptions of resident teaching skills confirmed the observed changes in CETs. Some CETs continued to practice and build their capacity for teaching after completing the rotation. Qualitatively, we derived four common themes among the data; communication, professional engagement, practice-based learning, and systems-based learning.

CONCLUSION

Resident teaching capacity was enriched after completing the clinical educator rotation. Other benefits included: enhanced patient communication and education, increased resident confidence, personal satisfaction with training, work life-balance and enhanced career satisfaction. Future research should focus on curricular content, faculty development, and delivery assessment. In addition, research efforts should identify appropriate emerging technologies to include in the curriculum for enhancing teaching capacity.

Variables associated with completeness of medical record documentation in the emergency department

OBJECTIVE

The completeness of ED medical record documentation is often suboptimal. We aimed to determine the variables associated with documentation completeness in a large, tertiary referral ED.

METHODS

We audited 1200 randomly selected medical records of patients who presented with either abdominal pain, cardiac chest pain, shortness of breath or headache between May-July 2013 and May-July 2016. Data were collected on patient and treating doctor variables. Documentation completeness was assessed using a 0-10 point scoring tool designed for the study. A maximum score was achieved if each of 10 pre-determined important items, specific to the presenting complaint, were documented (five medical history items, five physical examination items). Data were analysed using multivariate regression.

RESULTS

The presenting year, day and time, patient age and gender, preferred language, interpreter requirement, discharge destination and doctor gender were not associated with documentation completeness (P > 0.05). Patients with triage category 3 or pain score of 6-7 had higher documentation scores (P < 0.05). Compared to interns, registrars (effect size -0.72, 95% CI -1.02 to -0.42, P < 0.01) and consultants (-1.62, 95% CI -1.95 to -1.29, P < 0.01) scored significantly less. The headache patient subgroup scored significantly less than the other patient subgroups (-0.35, 95% CI -0.63 to -0.08, P = 0.01). For all presenting complaint subgroups, examination findings were less well documented than history items (P < 0.001).

CONCLUSION

Documentation completeness is less among senior doctors, headache patients and for examination findings. Research should determine if the supervision responsibilities of senior doctors affects documentation and if medico-legal and patient care implications exist.

Electronic Health Records as an Educational Tool: Viewpoint

BACKGROUND

Electronic health records (EHRs) have been adopted by most hospitals and medical offices in the United States. Because of the rapidity of implementation, health care providers have not been able to leverage the full potential of the EHR for enhancing clinical care, learning, and teaching. Physicians are spending an average of 49% of their working hours on EHR documentation, chart review, and other indirect tasks related to patient care, which translates into less face time with patients.

OBJECTIVE

The purpose of this article is to provide a preliminary framework to guide the use of EHRs in teaching and evaluation of residents.

METHODS

First we discuss EHR educational capabilities that have not been reviewed in sufficient detail in the literature and expand our discussion for each educational activity with examples. We emphasize quality improvement of clinical notes as a basic foundational skill using a spreadsheet-based application as an assessment tool. Next, we integrate the six Accreditation Council for Graduate Medical Education (ACGME) Core Competencies and Milestones (CCMs) framework with the Reporter-Interpreter-Manager-Educator (RIME) model to expand our assessments of other areas of resident performance related to EHR use. Finally, we discuss how clinical utility, clinical outcome, and clinical reasoning skills can be assessed in the EHR.

RESULTS

We describe a pilot conceptual framework-CCM framework-to guide and demonstrate the use of the EHR for education in a clinical setting.

CONCLUSIONS

As EHRs and other supporting technologies evolve, medical educators should continue to look for new opportunities within the EHR for education. Our framework is flexible to allow adaptation and use in most training programs. Future research should assess the validity of such methods on trainees' education.

A Heuristic Evaluation to Assess Use of After Visit Summaries for Supporting Continuity of Care

BACKGROUND

Outpatient providers often do not receive discharge summaries from acute care providers prior to follow-up visits. These outpatient providers may use the after-visit summaries (AVS) that are given to patients to obtain clinical information. It is unclear how effectively AVS support care coordination between clinicians.

OBJECTIVES

Goals for this effort include: (1) developing usability heuristics that may be applied both for assessment and to guide generation of medical documents in general, (2) conducting a heuristic evaluation to assess the use of AVS for communication between clinicians, and (3) providing recommendations for generating AVS that effectively support both patient/caregiver use and care coordination.

METHODS

We created a 17-item heuristic evaluation instrument for assessing usability of medical documents. Eight experts used the instrument to assess each of four simulated AVS. The simulations were created using examples from two hospitals and two pediatric patient cases developed by the National Institute of Standards and Technology.

RESULTS

Experts identified 224 unique usability problems ranging in severity from mild to catastrophic. Content issues (e.g., missing medical history, marital status of a 2-year-old) were rated as most severe, but widespread formatting and structural problems (e.g., inconsistent indentation, fonts, and headings; confusing ordering of information) were so distracting that they significantly reduced readers' ability to efficiently use the documents. Overall, issues in the AVS from Hospital 2 were more severe than those in the AVS from Hospital 1.

CONCLUSION

The new instrument allowed for quick, inexpensive evaluations of AVS. Usability issues such as unnecessary information, poor organization, missing information, and inconsistent formatting make it hard for patients, caregivers, and clinicians to use the AVS. The heuristics in the new instrument may be used as guidance to adapt electronic health record systems so that they generate more useful and usable medical documents.

Simulation of patient flow in multiple healthcare units using process and data mining techniques for model identification

INTRODUCTION

An approach to building a hybrid simulation of patient flow is introduced with a combination of data-driven methods for automation of model identification. The approach is described with a conceptual framework and basic methods for combination of different techniques. The implementation of the proposed approach for simulation of the acute coronary syndrome (ACS) was developed and used in an experimental study.

METHODS

A combination of data, text, process mining techniques, and machine learning approaches for the analysis of electronic health records (EHRs) with discrete-event simulation (DES) and queueing theory for the simulation of patient flow was proposed. The performed analysis of EHRs for ACS patients enabled identification of several classes of clinical pathways (CPs) which were used to implement a more realistic simulation of the patient flow. The developed solution was implemented using Python libraries (SimPy, SciPy, and others).

RESULTS

The proposed approach enables more a realistic and detailed simulation of the patient flow within a group of related departments. An experimental study shows an improved simulation of patient length of stay for ACS patient flow obtained from EHRs in Almazov National Medical Research Centre in Saint Petersburg, Russia.

CONCLUSION

The proposed approach, methods, and solutions provide a conceptual, methodological, and programming framework for the implementation of a simulation of complex and diverse scenarios within a flow of patients for different purposes: decision making, training, management optimization, and others.

[Effectiveness of an assessment of computerised medical records in a university hospital]

OBJECTIVE

The objective of this study was to evaluate a sample of electronic medical records (EMR) that was sufficiently representative of the different areas of care in a university hospital, as well as to verify the effectiveness of an initial intervention through a second evaluation.

METHODS

Medical records audits were performed in 2012 and 2013 by a blind peer review of random samples of care episodes, proportional to the activity of each clinical department, and with the same evaluation method being applied to all of them.

RESULTS

More than 1,000 episodes of care were reviewed in the 2audits. A significant improvement was found in hospital admissions (P=.000) in all the sections of the EMR analysed (P=.002), and was especially significant for the reason for consultation, for which its completion increased by 8.5% (p<.05), and also in the sections of the current process record (7.1%), physical examination (4.7%), allergies (3.9%), and clinical course (3.6%). The assessment of the discharge report, as a whole, showed an improvement (P=.001). In outpatient follow-up visits, a significant positive improvement was observed in the 4sections evaluated (P<.05), and also overall (P=.000).

CONCLUSIONS

According to study conditions, the dissemination of the results was effective in improving the quality of the EMR. The results have made it possible to implement actions to review the work processes in certain departments, and also the partial redesign of the interface on being a reproducible methodology accepted by the organisation.

Use of dictation as a tool to decrease documentation errors in electronic health records

Background: Use of Electronic Health Records is increasing. Copy-and-paste function is frequently used with higher rates of documentation errors. Studies to determine the nature of such errors are needed.Objectives: Determination of the effect of implementing a dictation system for completing notes on the quality of clinical documentation. We hypothesized that implementation of the dictation system for note writing would decrease the rate of errors in the progress notes as well as decrease the rate of copying and pasting. Design/Methods: A prospective interventional study in inpatient medical service for six months’ duration starting in July 2016. Resident physicians’ charts were reviewed by the attending physician on a daily basis. This study was done in a community based hospital affiliated to a university program. Residents’ physicians included Internal Medicine, Transitional year and Combined Internal Medicine Pediatrics residents. Charts reviewed for hospitalized patients. A total of 54 residents were offered a pre-intervention survey indicating their subjective use of copy/paste function. Response rate of 85.18%. Progress notes were reviewed on a daily basis for residents on their inpatient rotation. A total of 621 notes were reviewed. Results: Percentage of notes copied prior to the intervention was 92.73% which decreased to 49.71% post-intervention (RR of 0.54, 95% CI 0.48 0.60 Z statistic 11.005 with p-value &lt;0.0001). Of the copied notes percentage of errors pre-intervention was 58% with no errors identified post-intervention (RR of 0.005, 95% CI 0.0003 0.0795 Z statistic 3.752 with p-value 0.0002). Most of the errors are from notes copied by the same author (85.8%). The most common documentation error was in the physical examination section. Conclusion: Implementing a dictation system eliminated documentation errors over our six months’ study. Further studies are needed to check long effects of using such systems on documentation errors

Promoting Responsible Electronic Documentation: Validity Evidence for a Checklist to Assess Progress Notes in the Electronic Health Record

Construct: We aimed to develop an instrument to measure the quality of inpatient electronic health record- (EHR-) generated progress notes without requiring raters to review the detailed chart or know the patient.

BACKGROUND

Notes written in EHRs have generated criticism for being unnecessarily long and redundant, perpetuating inaccuracy and obscuring providers' clinical reasoning. Available assessment tools either focus on outpatient progress notes or require chart review by raters to develop familiarity with the patient.

APPROACH

We used medical literature, local expert review, and attending focus groups to develop and refine an instrument to evaluate inpatient progress notes. We measured interrater reliability and scored the selected-response elements of the checklist for a sample of 100 progress notes written by PGY-1 trainees on the general medicine service.

RESULTS

We developed an instrument with 18 selected-response items and four open-ended items to measure the quality of inpatient progress notes written in the EHR. The mean Cohen's kappa coefficient demonstrated good agreement at .67. The mean note score was 66.9% of maximum possible points (SD = 10.6, range = 34.4%-93.3%).

CONCLUSIONS

We present validity evidence in the domains of content, internal structure, and response process for a new checklist for rating inpatient progress notes. The scored checklist can be completed in approximately 7 minutes by a rater who is not familiar with the patient and can be done without extensive chart review. We further demonstrate that trainee notes show substantial room for improvement.

The 9-Item Physician Documentation Quality Instrument (PDQI-9) score is not useful in evaluating EMR (scribe) note quality in Emergency Medicine

BACKGROUND

Scribes are assisting Emergency Physicians by writing their electronic clinical notes at the bedside during consultations. They increase physician productivity and improve their working conditions. The quality of Emergency scribe notes is unevaluated and important to determine.

OBJECTIVE

The primary objective of the study was to determine if the quality of Emergency Department scribe notes was equivalent to physician only notes, using the Physician Documentation Quality Instrument, Nine-item tool (PDQI-9).

METHODS

This was a retrospective, observational study comparing 110 scribed to 110 non-scribed Emergency Physician notes written at Cabrini Emergency Department, Australia. Consultations during a randomised controlled trial of scribe/doctor productivity in 2016 were used. Emergency physicians and nurses rated randomly selected, blinded and de-identified notes, 2 raters per note. Comparisons were made between paired scribed and unscribed notes and between raters of each note. Characteristics of individual raters were examined. The ability of the tool to discriminate between good and poor notes was tested.

RESULTS

The PDQI-9 tool has significant issues. Individual items had good internal consistency (Cronbach's alpha=0.93), but there was very poor agreement between raters (Pearson's r=0.07, p=0.270). There were substantial differences in PDQI-9 scores allocated by each rater, with some giving typically lower scores than others, F(25,206)=1.93, p=0.007. The tool was unable to distinguish good from poor notes, F(3,34)=1.15, p=0.342. There was no difference in PDQI-9 score between scribed and non-scribed notes.

CONCLUSIONS

The PDQI-9 documentation quality tool did not demonstrate reliability or validity in evaluating Emergency Medicine consultation notes. We found no evidence that scribed notes were of poorer quality than non-scribed notes, however Emergency scribe note quality has not yet been determined.

Clinical Natural Language Processing in 2014: Foundational Methods Supporting Efficient Healthcare

OBJECTIVE

To summarize recent research and present a selection of the best papers published in 2014 in the field of clinical Natural Language Processing (NLP).

METHOD

A systematic review of the literature was performed by the two section editors of the IMIA Yearbook NLP section by searching bibliographic databases with a focus on NLP efforts applied to clinical texts or aimed at a clinical outcome. A shortlist of candidate best papers was first selected by the section editors before being peer-reviewed by independent external reviewers.

RESULTS

The clinical NLP best paper selection shows that the field is tackling text analysis methods of increasing depth. The full review process highlighted five papers addressing foundational methods in clinical NLP using clinically relevant texts from online forums or encyclopedias, clinical texts from Electronic Health Records, and included studies specifically aiming at a practical clinical outcome. The increased access to clinical data that was made possible with the recent progress of de-identification paved the way for the scientific community to address complex NLP problems such as word sense disambiguation, negation, temporal analysis and specific information nugget extraction. These advances in turn allowed for efficient application of NLP to clinical problems such as cancer patient triage. Another line of research investigates online clinically relevant texts and brings interesting insight on communication strategies to convey health-related information.

CONCLUSIONS

The field of clinical NLP is thriving through the contributions of both NLP researchers and healthcare professionals interested in applying NLP techniques for concrete healthcare purposes. Clinical NLP is becoming mature for practical applications with a significant clinical impact.

Duly noted: Lessons from a two-site intervention to assess and improve the quality of clinical documentation in the electronic health record

BACKGROUND

Communication errors are identified as a root cause contributing to a majority of sentinel events. The clinical note is a cornerstone of physician communication, yet there are few published interventions on teaching note writing in the electronic health record (EHR). This is a prospective, two-site, quality improvement project to assess and improve the quality of clinical documentation in the EHR using a validated assessment tool.

METHODS

Internal Medicine (IM) residents at the University of Kentucky College of Medicine (UK) and Montefiore Medical Center/Albert Einstein College of Medicine (MMC) received one of two interventions during an inpatient ward month: either a lecture, or a lecture and individual feedback on progress notes. A third group of residents in each program served as control. Notes were evaluated with the Physician Documentation Quality Instrument 9 (PDQI-9).

RESULTS

Due to a significant difference in baseline PDQI-9 scores at MMC, the sites were not combined. Of 75 residents at the UK site, 22 were eligible, 20 (91%) enrolled, 76 notes in total were scored. Of 156 residents at MMC, 22 were eligible, 18 (82%) enrolled, 40 notes in total were scored. Note quality did not improve as measured by the PDQI-9.

CONCLUSION

This educational quality improvement project did not improve the quality of clinical documentation as measured by the PDQI-9. This project underscores the difficulty in improving note quality. Further efforts should explore more effective educational tools to improve the quality of clinical documentation in the EHR.

An electronic documentation system improves the quality of admission notes: a randomized trial

OBJECTIVE

There are concerns that structured electronic documentation systems can limit expressivity and encourage long and unreadable notes. We assessed the impact of an electronic clinical documentation system on the quality of admission notes for patients admitted to a general medical unit.

METHODS

This was a prospective randomized crossover study comparing handwritten paper notes to electronic notes on different patients by the same author, generated using a semistructured electronic admission documentation system over a 2-month period in 2014. The setting was a 4-team, 80-bed general internal medicine clinical teaching unit at a large urban academic hospital. The quality of clinical documentation was assessed using the QNOTE instrument (best possible score = 100), and word counts were assessed for free-text sections of notes.

RESULTS

Twenty-one electronic-paper note pairs (42 notes) written by 21 authors were randomly drawn from a pool of 303 eligible notes. Overall note quality was significantly higher in electronic vs paper notes (mean 90 vs 69, P < .0001). The quality of free-text subsections (History of Present Illness and Impression and Plan) was significantly higher in the electronic vs paper notes (mean 93 vs 78, P < .0001; and 89 vs 77, P = .001, respectively). The History of Present Illness subsection was significantly longer in electronic vs paper notes (mean 172.4 vs 92.4 words, P = .0001).

CONCLUSIONS

An electronic admission documentation system improved both the quality of free-text content and the overall quality of admission notes. Authors wrote more in the free-text sections of electronic documents as compared to paper versions.

Computer versus physician identification of gastrointestinal alarm features

OBJECTIVE

It is important for clinicians to inquire about "alarm features" as it may identify those at risk for organic disease and who require additional diagnostic workup. We developed a computer algorithm called Automated Evaluation of Gastrointestinal Symptoms (AEGIS) that systematically collects patient gastrointestinal (GI) symptoms and alarm features, and then "translates" the information into a history of present illness (HPI). Our study's objective was to compare the number of alarms documented by physicians during usual care vs. that collected by AEGIS.

METHODS

We performed a cross-sectional study with a paired sample design among patients visiting adult GI clinics. Participants first received usual care by their physicians and then completed AEGIS. Each individual thus contributed both a physician-documented and computer-generated HPI. Blinded physician reviewers enumerated the positive alarm features (hematochezia, melena, hematemesis, unintentional weight loss, decreased appetite, and fevers) mentioned in each HPI. We compared the number of documented alarms within patient using the Wilcoxon signed-rank test.

RESULTS

Seventy-five patients had both physician and AEGIS HPIs. AEGIS identified more patients with positive alarm features compared to physicians (53% vs. 27%; p<.001). AEGIS also documented more positive alarms (median 1, interquartile range [IQR] 0-2) vs. physicians (median 0, IQR 0-1; p<.001). Moreover, clinicians documented only 30% of the positive alarms self-reported by patients through AEGIS.

CONCLUSIONS

Physicians documented less than one-third of red flags reported by patients through a computer algorithm. These data indicate that physicians may under report alarm features and that computerized "checklists" could complement standard HPIs to bolster clinical care.

The effectiveness of a bundled intervention to improve resident progress notes in an electronic health record

Providers nationally have observed a decline in the quality of documentation after implementing electronic health records (EHRs). In this pilot study, we examined the effectiveness of an intervention bundle designed to improve resident progress notes written in an EHR and to establish the reliability of an audit tool used to evaluate notes. The bundle consisted of establishing note-writing guidelines, developing an aligned note template, and educating interns about the guidelines and using the template. Twenty-five progress notes written by pediatric interns before and after this intervention were examined using an audit tool. Reliability of the tool was evaluated using the intraclass correlation coefficient (ICC). The total score of the audit tool was summarized in terms of means and standard deviation. Individual item responses were summarized using percentages and compared between the pre- and postintervention assessment using the Fisher exact test. The ICC for the audit tool was 0.96 (95% confidence interval: 0.91-0.98). A significant improvement in the total note score and in questions related to note clutter was seen. No significant improvement was seen for questions related to copy-paste. The study suggests that an intervention bundle can lead to some improvements in note writing.

Electronic health records improve clinical note quality

BACKGROUND AND OBJECTIVE

The clinical note documents the clinician's information collection, problem assessment, clinical management, and its used for administrative purposes. Electronic health records (EHRs) are being implemented in clinical practices throughout the USA yet it is not known whether they improve the quality of clinical notes. The goal in this study was to determine if EHRs improve the quality of outpatient clinical notes.

MATERIALS AND METHODS

A five and a half year longitudinal retrospective multicenter quantitative study comparing the quality of handwritten and electronic outpatient clinical visit notes for 100 patients with type 2 diabetes at three time points: 6 months prior to the introduction of the EHR (before-EHR), 6 months after the introduction of the EHR (after-EHR), and 5 years after the introduction of the EHR (5-year-EHR). QNOTE, a validated quantitative instrument, was used to assess the quality of outpatient clinical notes. Its scores can range from a low of 0 to a high of 100. Sixteen primary care physicians with active practices used QNOTE to determine the quality of the 300 patient notes.

RESULTS

The before-EHR, after-EHR, and 5-year-EHR grand mean scores (SD) were 52.0 (18.4), 61.2 (16.3), and 80.4 (8.9), respectively, and the change in scores for before-EHR to after-EHR and before-EHR to 5-year-EHR were 18% (p<0.0001) and 55% (p<0.0001), respectively. All the element and grand mean quality scores significantly improved over the 5-year time interval.

CONCLUSIONS

The EHR significantly improved the overall quality of the outpatient clinical note and the quality of all its elements, including the core and non-core elements. To our knowledge, this is the first study to demonstrate that the EHR significantly improves the quality of clinical notes.