Structured Reporting of Medical Findings: Evaluation of a System in Gastroenterology

Assisted documentation as a new focus for artificial intelligence in endoscopy: the precedent of reliable withdrawal time and image reporting

BACKGROUND : Reliable documentation is essential for maintaining quality standards in endoscopy; however, in clinical practice, report quality varies. We developed an artificial intelligence (AI)-based prototype for the measurement of withdrawal and intervention times, and automatic photodocumentation. METHOD:  A multiclass deep learning algorithm distinguishing different endoscopic image content was trained with 10 557 images (1300 examinations, nine centers, four processors). Consecutively, the algorithm was used to calculate withdrawal time (AI prediction) and extract relevant images. Validation was performed on 100 colonoscopy videos (five centers). The reported and AI-predicted withdrawal times were compared with video-based measurement; photodocumentation was compared for documented polypectomies. RESULTS:  Video-based measurement in 100 colonoscopies revealed a median absolute difference of 2.0 minutes between the measured and reported withdrawal times, compared with 0.4 minutes for AI predictions. The original photodocumentation represented the cecum in 88 examinations compared with 98/100 examinations for the AI-generated documentation. For 39/104 polypectomies, the examiners' photographs included the instrument, compared with 68 for the AI images. Lastly, we demonstrated real-time capability (10 colonoscopies). CONCLUSION : Our AI system calculates withdrawal time, provides an image report, and is real-time ready. After further validation, the system may improve standardized reporting, while decreasing the workload created by routine documentation.

Computer-Based Intelligent Solutions for the Diagnosis of Gastroesophageal Reflux Disease Phenotypes and Chicago Classification 3.0

Gastroesophageal reflux disease (GERD) is a multidisciplinary disease; therefore, when treating GERD, a large amount of data needs to be monitored and managed.The aim of our study was to develop a novel automation and decision support system for GERD, primarily to automatically determine GERD and its Chicago Classification 3.0 (CC 3.0) phenotypes. However, phenotyping is prone to errors and is not a strategy widely known by physicians, yet it is very important in patient treatment. In our study, the GERD phenotype algorithm was tested on a dataset with 2052 patients and the CC 3.0 algorithm was tested on a dataset with 133 patients. Based on these two algorithms, a system was developed with an artificial intelligence model for distinguishing four phenotypes per patient. When a physician makes a wrong phenotyping decision, the system warns them and provides the correct phenotype. An accuracy of 100% was obtained for both GERD phenotyping and CC 3.0 in these tests. Finally, since the transition to using this developed system in 2017, the annual number of cured patients, around 400 before, has increased to 800. Automatic phenotyping provides convenience in patient care, diagnosis, and treatment management. Thus, the developed system can substantially improve the performance of physicians.

The Birth and Evolution of a Discipline Devoted to Information in Biomedicine and Health Care

Background: The journal Methods of Information in Medicine, founded in 1962, has now completed its 50th volume. Its publications during the last five decades reflect the formation of a discipline that deals with information in biomedicine and health care. Objectives: To report about 1) the journal‘s origin, 2) the individuals who have significantly contributed to it, 3) trends in the journal’s aims and scope, 4) influential papers and 5) major topics published in Methods over the years.

Methods: Methods included analysing the correspondence and journal issues in the archives of the editorial office and of the publisher, citation analysis using the ISI and Scopus databases, and analysing the articles’ Medical Subject Headings (MeSH) in MEDLINE.

Results: In the journal’s first 50 years 208 editorial board members and/or editors contributed to the journal’s development, with most individuals coming from Europe and North America. The median time of service was 11 years. At the time of analysis 2,456 articles had been indexed with Me SH. Topics included computerized systems of various types, informatics methodologies, and topics related to a specific medical domain. Some MeSH topic entries were heavily and regularly represented in each of the journal‘s five decades (e.g. information systems and medical records), while others were important in a particular decade, but not in other decades (e.g. punched-card systems and systems integration). Seven papers were cited more than 100 times and these also covered a broad range of themes such as knowledge representation, analysis of biomedical data and knowledge, clinical decision support and electronic patient records. Conclusions: Methods of Information in Medicine is the oldest international journal in biomedical informatics. The journal’s development over the last 50 years correlates with the formation of this new discipline. It has and continues to stress the basic methodology and scientific fundamentals of organizing, representing and analysing data, information and knowledge in biomedicine and health care. It has and continues to stimulate multi-disciplinary communication on research that is devoted to high-quality, efficient health care, to quality of life and to the progress of biomedicine and the health sciences.

Development, implementation, and evaluation of a structured reporting web tool for abdominal aortic aneurysms

BACKGROUND

The majority of radiological reports are lacking a standard structure. Even within a specialized area of radiology, each report has its individual structure with regards to details and order, often containing too much of non-relevant information the referring physician is not interested in. For gathering relevant clinical key parameters in an efficient way or to support long-term therapy monitoring, structured reporting might be advantageous.

OBJECTIVE

Despite of new technologies in medical information systems, medical reporting is still not dynamic. To improve the quality of communication in radiology reports, a new structured reporting system was developed for abdominal aortic aneurysms (AAA), intended to enhance professional communication by providing the pertinent clinical information in a predefined standard.

METHODS

Actual state analysis was performed within the departments of radiology and vascular surgery by developing a Technology Acceptance Model. The SWOT (strengths, weaknesses, opportunities, and threats) analysis focused on optimization of the radiology reporting of patients with AAA. Definition of clinical parameters was achieved by interviewing experienced clinicians in radiology and vascular surgery. For evaluation, a focus group (4 radiologists) looked at the reports of 16 patients. The usability and reliability of the method was validated in a real-world test environment in the field of radiology.

RESULTS

A Web-based application for radiological "structured reporting" (SR) was successfully standardized for AAA. Its organization comprises three main categories: characteristics of pathology and adjacent anatomy, measurements, and additional findings. Using different graphical widgets (eg, drop-down menus) in each category facilitate predefined data entries. Measurement parameters shown in a diagram can be defined for clinical monitoring and be adducted for quick adjudications. Figures for optional use to guide and standardize the reporting are embedded. Analysis of variance shows decreased average time required with SR to obtain a radiological report compared to free-text reporting (P=.0001). Questionnaire responses confirm a high acceptance rate by the user.

CONCLUSIONS

The new SR system may support efficient radiological reporting for initial diagnosis and follow-up for AAA. Perceived advantages of our SR platform are ease of use, which may lead to more accurate decision support. The new system is open to communicate not only with clinical partners but also with Radiology Information and Hospital Information Systems.

Method of electronic health record documentation and quality of primary care

OBJECTIVE

Physicians who more intensively interact with electronic health records (EHRs) through their documentation style may pay greater attention to coded fields and clinical decision support and thus may deliver higher quality care. We measured the quality of care of physicians who used three predominating EHR documentation styles: dictation, structured documentation, and free text.

METHODS

We conducted a retrospective analysis of visits by patients with coronary artery disease and diabetes to the Partners Primary Care Practice Based Research Network. The main outcome measures were 15 EHR-based coronary artery disease and diabetes measures assessed 30 days after primary care visits.

RESULTS

During the 9-month study period, 7000 coronary artery disease and diabetes patients made 18 569 visits to 234 primary care physicians of whom 20 (9%) predominantly dictated their notes, 68 (29%) predominantly used structured documentation, and 146 (62%) predominantly typed free text notes. In multivariable modeling adjusted for clustering by patient and physician, quality of care appeared significantly worse for dictators than for physicians using the other two documentation styles on three of 15 measures (antiplatelet medication, tobacco use documentation, and diabetic eye exam); better for structured documenters for three measures (blood pressure documentation, body mass index documentation, and diabetic foot exam); and better for free text documenters on one measure (influenza vaccination). There was no measure for which dictators had higher quality of care than physicians using the other two documentation styles.

CONCLUSIONS

EHR-assessed quality is necessarily documentation-dependent, but physicians who dictated their notes appeared to have worse quality of care than physicians who used structured EHR documentation.

CLINICAL TRIAL REGISTRATION NUMBER

ClinicalTrials.gov Identifier: NCT00235040.

Documentation quality and time costs

The Department of Veterans Affairs (VA) performs over 800,000 disability exams and distributes over $37 billion in disability benefits per year. VA developed and deployed a computer-based disability exam documentation system in order to improve exam report quality and timeliness. We conducted a randomized controlled trial comparing joint disability examinations supported by computerized templates to the examinations documented via dictation, to determine if the system met the intended goals or had unintended consequences. Consenting veterans were randomized to undergo exams documented using computerized templates or via dictation. We compared exam report quality, documentation time costs, encounter length, total time to fulfill an exam request with a finalized exam report, and veteran satisfaction. Computer-based templates resulted in disability exam reports that had higher quality scores (p. 0.042) and were returned to the requesting office faster than exam reports created via dictation (p. 0.02).

Documentation time and veteran satisfaction were similar for both the documentation techniques. Encounter length was significantly longer for the template group. Computer-based templates impacted the VA disability evaluation system by improving report quality scores and production time and lengthening encounter times. Oversight bodies have called for mandated use of computer-based templates nationwide. We believe mandates regarding use of health information technology should be guided by data regarding its positive and negative impacts.

Development of an electronic web-based software for the management of colorectal cancer target referral patients

In response to concern about lengthy waiting times for cancer treatment in the UK, the Department of Health introduced 'the colorectal cancer target referral scheme' to improve the referral process for suspected cancer. A user-centred web-based intranet software was developed reflecting the core work of the multi-disciplinary cancer team and the patient journey. The method used was primarily based on the concept of involving the end users (clinicians, nurses, administration staff) in the process of problem definition, software design, formative evaluation, development and implementation, from the very beginning, to ensure its relevance, functionality, and effectiveness. This software improved the interdisciplinary communication among doctors. All patients met the government waiting targets and proved to be a facilitative tool for audit, research and further prospective assessment of our service. Implementing a functional software design is mandatory for the management of target referral patients.

Development of electronic software for the management of trauma patients on the orthopaedic unit

INTRODUCTION

Continuity of patient care is an essential prerequisite for the successful running of a trauma surgery service. This is becoming increasingly difficult because of the new working arrangements of junior doctors. Handover is now central to ensure continuity of care following shift change over. The purpose of this study was to compare the quality of information handed over using the traditional ad hoc method of a handover sheet versus a web-based electronic software programme. It was hoped that through improved quality of handover the new system would have a positive impact on clinical care, risk and time management.

METHODS

Data was prospectively collected and analyzed using the SPSS 14 statistical package. The handover data of 350 patients using a paper-based system was compared to the data of 357 cases using the web-based system. Key data included basic demographic data, responsible surgeon, location of patient, injury site including site, whether fractures were open or closed, concomitant injuries and the treatment plan. A survey was conducted amongst health care providers to assess the impact of the new software.

RESULTS

With the introduction of the electronic handover system, patients with missing demographic data reduced from 35.1% to 0.8% (p<0.0001) and missing patient location from 18.6% to 3.6% (p<0.0001). Missing consultant information and missing diagnosis dropped from 12.9% to 2.0% (p<0.0001) and from 11.7% to 0.8% (p<0.0001), respectively. The missing information regarding side and anatomical site of the injury was reduced from 31.4% to 0.8% (p<0.0001) and from 13.7% to 1.1% (p<0.0001), respectively. In 96.6% of paper ad hoc handovers it was not stated whether the injury was 'closed' or 'open', whereas in the electronic group this information was evident in all 357 patients (p<0.0001). A treatment plan was included only in 52.3% of paper handovers compared to 94.7% (p<0.0001) of electronic handovers. A survey revealed 96% of members of the trauma team felt an improvement of handover since the introduction of the software, and 94% of members were satisfied with the software.

CONCLUSIONS

The findings of our study show that the use of web-based electronic software is effective in facilitating and improving the quality of information passed during handover. Structured software also aids in improving work flow amongst the trauma team. We argue that an improvement in the quality of handover is an improvement in clinical practice.

Checking concordance between findings and diagnoses in sonographic reports by a knowledge-based documentation system

PURPOSE

Sonographic reports are examiner-dependent and may not always be reliable. We investigated concordance between documented findings and diagnostic conclusions--not the objective correctness of both--with the help of a knowledge-based documentation system.

MATERIALS AND METHODS

The knowledge-based documentation system SonoConsult (SC) is routinely used in the ultrasound unit of a gastroenterological clinic for more than four years. Physicians documented findings with goal directed questionnaires, and diagnostic conclusions with free text. The consistency of documented findings and diagnoses was checked with the help of SC in a two-step process: 1. the diagnoses inferred by SC based on the documented findings were compared to the diagnoses of the physicians stated as free text. 2. In case of discrepancies, a more thorough comparison was performed manually by the medical authors of this study. For judging the practical relevance of discrepancies, diagnostic codes were pre-classified as a) being presumably of higher and lower relevance for the clinician and b) requiring simple or complex inference rules from the findings.

RESULTS

In a first series of 250 consecutive cases with 934 diagnoses (3.7 diagnoses per case), 71.1% showed agreement between diagnoses of the physicians and of SC and were judged as consistent compared to the documented findings. 24.4% of the diagnoses suggested by the documented findings, however, were not mentioned by the physicians (false negative) and 4.5% were mentioned by the physicians but not suggested by the documented findings (false positive). From the 24.4% missing diagnoses, 40% were pre-classified as being of higher relevance for the clinician. In a second series of 161 consecutive cases with 501 diagnoses (3.1 diagnoses per case), 61.1% were judged as consistent compared to the documented findings, 36.1% false negative and 2.8% false positive. In this study, we differentiated the missing diagnoses due to their inferential complexity: From the 152 complex diagnoses, 44% were missing, while from the 349 simple diagnoses, 32.7% were missing.

CONCLUSION

As shown for a sonographic department of a clinic of internal medicine, in sonographic reports, one has to be aware of discrepancies between question-set-based documentations of findings and diagnostic conclusions of the examiners. While a detailed documentation of findings is the basis of quality control, consistency checks between documented findings and diagnostic conclusions, which might be done automatically in an electronic patient record, would considerably improve the quality of the reports.

Computerized endoscopic reporting is no more time-consuming than reporting with conventional methods

BACKGROUND

Endoscopists use different methods for reporting their findings after a gastrointestinal endoscopy. These may result in handwritten, dictated, or computerized reports. The time needed to create the report is an important parameter for acceptance of the method used. It is also important to be aware of the possible advantages and disadvantages of these different methods. The aim of this study was to compare time aspects of different methods of report writing.

METHODS

Three different methods of report writing, i.e., handwritten, dictated, and computerized, were compared. In three different endoscopy departments, one investigator recorded the time needed to compose the report and to send it to the referring doctor. The time needed to describe different diagnoses at endoscopy was compared between the systems.

RESULTS

Handwritten reports were completed in an average time of 113 s, free text dictated reports by the endoscopist in 65 s with an additional 172 s allowed for the typist, and computerized, pre-defined reports were completed in 86 s. The incidences of abnormalities found in the reports of the different hospitals were comparable.

CONCLUSION

To a large extent, computerized, pre-defined reports could be composed in almost the same amount of time as handwritten and dictated reports. Free text dictated and computerized, pre-defined reports are both stored in the hospital information system, but only computerized, pre-defined reports including endoscopic pictures are stored in a structured database, which makes statistical analysis possible.

Electronic Templates versus Dictation for the Completion of Mohs Micrographic Surgery Operative Notes

BACKGROUND

Operative notes can be generated electronically by manual input of the entire note, free-form oral dictation, or using either an electronic template or a template for dictation. There are few studies that have directly compared these modalities in terms of speed, accuracy, and completeness.

OBJECTIVE

The objective was to determine whether electronic templates are more efficient and reduce errors compared to free-form oral dictation for the completion of Mohs micrographic surgery operative notes.

METHODS

Operative notes for 110 consecutive Mohs micrographic surgery cases were completed either by oral dictation or by electronic template. The time to dictate or complete the template was recorded for each note. Notes were subsequently edited, recording the number and type of errors as well as the time required to edit each note.

RESULTS

Compared with dictation, operative notes completed with the electronic template had fewer errors (5.8% vs. 81%), took less time to complete (175.5 seconds vs. 240.0 seconds), took less time to review and edit (41.6 seconds vs. 201.1 seconds), and were completed and signed in a more timely fashion (0.115 days vs. 20.7 days).

CONCLUSION

Electronic templates are a more accurate and rapid method compared to free-form oral dictation for the completion of Mohs micrographic surgery operative notes and have the advantage of being immediately available to review and sign.

Paper versus computer: feasibility of an electronic medical record in general pediatrics

BACKGROUND

Implementation of electronic medical record systems promises significant advances in patient care, because such systems enhance readability, availability, and data quality. Structured data entry (SDE) applications can prompt for completeness, provide greater accuracy and better ordering for searching and retrieval, and permit validity checks for data quality monitoring, research, and especially decision support. A generic SDE application (OpenSDE) to support documentation of patient history and physical examination findings was developed and tailored for the domain of general pediatrics.

OBJECTIVE

To evaluate OpenSDE for its completeness, uniformity of reporting, and usability in general pediatrics.

METHODS

Four (trainee) pediatricians documented data for 8 first-visit patients in the traditional, paper-based, medical record and immediately thereafter in OpenSDE (electronic record). The 32 paper records obtained served as the common data source for data entry in OpenSDE by the other 3 physicians (transcribed record). Data entered by 2 experienced users, with all patient information present in the paper record, served as the control record. Data entry times were recorded, and a questionnaire was used to assess users' experiences with OpenSDE.

RESULTS

Clinicians documented 44% of all available patient information identically in the paper and electronic records. Twenty-five percent of all patient information was documented only in the paper record, and 31% was present only in the electronic record. Differences were found in patient history and physical examination documentation in the electronic record; more information was missing for patient history (38%) than for physical examination (15%). Furthermore, physical examination contained more additional information (39%) than did patient history (21%). The interobserver agreement of documentation of patient information from the same data source was fair to moderate, with kappa values of 0.39 for patient history and 0.40 for physical examination. Data entry times in OpenSDE decreased from 25 minutes to <15 minutes, indicating a learning effect. The questionnaire revealed a positive attitude toward the use of OpenSDE in daily practice.

CONCLUSION

OpenSDE seems to be a promising application for the support of physician data entry in general pediatrics.

Informatics challenges for the impending patient information explosion

As we move toward an era when health information is more readily accessible and transferable, there are several issues that will arise. This article addresses the challenges of information filtering, context-sensitive decision support, legal and ethical guidelines regarding obligations to obtain and use the information, aligning patient and health professionals' expectations in regard to the use and usefulness of the information, and enhancing data reliability. The authors discuss the issues and offer suggestions for addressing them.

Image documentation of endoscopic findings in ulcerative colitis: photographs or video clips?

BACKGROUND

Previous studies have shown deficiencies in the endoscopy reports and substantial interobserver variation in the assessments of endoscopic findings. The aim of this study was to determine how to perform systematic digital image documentation in ulcerative colitis and to evaluate if mucosal inflammation is assessed equally on a still image and on a video clip.

METHODS

Eighteen video clips and their corresponding photographs that visualize different severities of ulcerative colitis were shown in randomized order to 20 experienced endoscopists. They assessed the mucosal inflammation of each image twice on a visual analog scale. Three comparisons were performed between the video clips, the photographs, and the video clips to the photographs, respectively.

RESULTS

The mean score of the inflammation of the video clips at tape 1 and 2 was 4.74: 95% confidence interval (CI)[4.41, 5.08] and 4.90: 95% CI[4.56, 5.24), respectively, and of the photographs 4.53: 95% CI[4.19, 4.88] and 4.43: 95% CI[4.09, 4.77], respectively. The first answer explains 83% of the variation in the second answer for all comparisons, and the agreement index ranged from 0.38 to 0.42.

CONCLUSIONS

The mucosal inflammation might be documented nearly as well with a still image as on a video clip. Systematic use of still images probably improves the endoscopy reports by adding more objective information about the mucosal inflammation.

Quality of colonoscopy reporting: a process of care study

OBJECTIVE

Several groups have developed guidelines for specific content necessary in endoscopic procedure reports. Little information is available assessing adherence to reporting recommendations, and little is known about common reporting errors. The aim of this study was to assess the quality of colonoscopy reporting and to identify possible areas of improvement.

METHODS

Using the 1997 American Society for GI Endoscopy guidelines for endoscopy reporting, we created operational definitions for adherence to each guideline. We then created 31 specific process of care criteria to assess adherence to each of these operational definitions. We subdivided the 31 specific process of care criteria into six domains: demographic information, patient history, sedation procedure, adequacy of preparation/visibility, lesion identification/removal, and procedure interpretation. Reports obtained from 122 separate endoscopy centers were reviewed for adherence to the guidelines. Adequate performance for any criterion was defined as 70% or better compliance.

RESULTS

Performance varied widely across the domains. Clinicians demonstrated adequate performance on sedation procedure (75%) and lesion identification/removal (84%). Clinicians scored poorly on demographic data (69%), patient history (57%), procedure quality (40%), and procedure interpretation (58%).

CONCLUSIONS

Clinicians' colonoscopy reporting practices are widely variable and often suboptimal. There is an opportunity to improve the quality of care in colonoscopy reporting by improving physicians' adherence to established standards.

Standardization in endoscopic reporting: implications for clinical practice and research

The large number of endoscopies and endoscopic reports produced in the United States represents a large repository of clinical data. However, reports are highly variable in content and structure and therefore cannot be used to create clinical databases. The introduction of automated endoscopic reporting systems should permit database creation only if acceptable standards for the structure and content of the report are developed. The structure of an endoscopic report is the framework in which the specific details of the endoscopy can be recorded. The basic components can consist of the following: Patient, Visit, Study, Result, Diagnosis, and Recommendation. Precise definition of each of these components requires consensus on what the minimum included elements should be. Experience with the Minimal Standard Terminology indicates that it is possible to create a broadly acceptable lexicon of descriptive endoscopic terms which can be included as a Result. The systematic development of the structure and content of endoscopic reports is mandatory before it is possible to create large, clinically useful databases of endoscopic reports.

Integration and communication for the continuity of cardiac care (I4C)

The project I4C (Integration and Communication for the Continuity of Cardiac Care) is carried out for the advancement of cardiac care, from prevention to follow-up. The goals of I4C are: (1) integrated access to patient data, wherever they are stored; (2) support of evidence-based care; (3) consistent recording of patient data (eg, patient history, electrocardiograms IECGs] or cine-angios) in a multimedia patient record; and (4) a documented reference data set for research. In several clinics, workstations are being installed to serve the four goals. Integration with other information systems in clinical care is realized by encapsulation. A computer-based patient record (ORCA) has been developed to support the collection, consultation, and sharing of patient data. In I4C, ORCA is intended for use in a research setting as well as routine patient care. The functionality of ORCA covers the collection of patient history data in a highly structured manner, the recording of drug prescriptions, an overview of laboratory test results, and viewers for ECGs and angiographic images. At present, structured data entry and consultation is supported in six European languages.

Standard Generalized Markup Language for self-defining structured reports

Structured reporting is the process of using standardized data elements and predetermined data-entry formats to record observations. The Standard Generalized Markup Language (SGML; International Standards Organization (ISO) 8879:1986)--an open, internationally accepted standard for document interchange was used to encode medical observations acquired in an Internet-based structured reporting system. The resulting report is self-documenting: it includes a definition of its allowable data fields and values encoded as a report-specific SGML document type definition (DTD). The data-entry forms, DTD, and report document instances are based on report specifications written in a simple, SGML-based language designed for that purpose. Reporting concepts can be linked with those of external vocabularies such as the Unified Medical Language System (UMLS) Metathesaurus. The use of open standards such as SGML is an important step in the creation of open, universally comprehensible structured reports.

Medical narratives in electronic medical records

In this article, we describe the state of the art and directions of current development and research with respect to the inclusion of medical narratives in electronic medical-record systems. We used information about 20 electronic medical-record systems as presented in the literature. We divided these systems into 'classical' systems that matured before 1990 and are now used in a broad range of medical domains, and 'experimental' systems, more recently developed and, in general, more innovative. In the literature, three major challenges were addressed: facilitation of direct data entry, achieving unambiguous understandability of data, and improvement of data presentation. Promising approaches to tackle the first and second challenge are the use of dynamic data-entry forms that anticipate sensible input, and free-text data entry followed by natural-language interpretation. Both these approaches require a highly expressive medical terminology. How to facilitate the access to medical narratives has not been studied much. We found facilitating examples of presenting this information as fluent prose, of optimising the screen design with fixed position cues, and of imposing medical narratives with a structure of indexable paragraphs that can be used in flowsheets. We conclude that further study is needed to develop an optimal searching structure for medical narratives.

Accuracy of data in computer-based patient records

Data in computer-based patient records (CPRs) have many uses beyond their primary role in patient care, including research and health-system management. Although the accuracy of CPR data directly affects these applications, there has been only sporadic interest in, and no previous review of, data accuracy in CPRs. This paper reviews the published studies of data accuracy in CPRs. These studies report highly variable levels of accuracy. This variability stems from differences in study design, in types of data studied, and in the CPRs themselves. These differences confound interpretation of this literature. We conclude that our knowledge of data accuracy in CPRs is not commensurate with its importance and further studies are needed. We propose methodological guidelines for studying accuracy that address shortcomings of the current literature. As CPR data are used increasingly for research, methods used in research databases to continuously monitor and improve accuracy should be applied to CPRs.