Informatics and the American College of Surgeons National Surgical Quality Improvement Program: automated processes could replace manual record review

Graphical analysis of guideline adherence to detect systemwide anomalies in HIV diagnostic testing

Background

Analyses of electronic medical databases often compare clinical practice to guideline recommendations. These analyses have a limited ability to simultaneously evaluate many interconnected medical decisions. We aimed to overcome this limitation with an alternative method and apply it to the diagnostic workup of HIV, where misuse can contribute to HIV transmission, delay care, and incur unnecessary costs.

Methods

We used graph theory to assess patterns of HIV diagnostic testing in a national healthcare system. We modeled the HIV diagnostic testing guidelines as a directed graph. Each node in the graph represented a test, and the edges pointed from one test to the next in chronological order. We then graphed each patient’s HIV testing. This set of patient-level graphs was aggregated into a single graph. Finally, we compared the two graphs, the first representing the recommended approach to HIV diagnostic testing and the second representing the observed patterns of HIV testing, to assess for clinical practice deviations.

Results

The HIV diagnostic testing of 1.643 million patients provided 8.790 million HIV diagnostic test results for analysis. Significant deviations from recommended practice were found including the use of HIV resistance tests (n = 3,007) and HIV nucleic acid tests (n = 16,567) instead of the recommended HIV screen.

Conclusions

We developed a method that modeled a complex medical scenario as a directed graph. When applied to HIV diagnostic testing, we identified deviations in clinical practice from guideline recommendations. The model enabled the identification of intervention targets and prompted systemwide policy changes to enhance HIV detection.

A Keyword Approach to Identify Adverse Events Within Narrative Documents From 4 Italian Institutions

OBJECTIVES

Existing methods for measuring adverse events in hospitals intercept a restricted number of events. Text mining refers to a range of techniques to extract data from narrative sources. The goal of this study was to evaluate the performance of an automated approach for extracting adverse event keywords from within electronic health records.

METHODS

The study involved 4 medical centers in the Region of Lombardy. A starting set of keywords was trained in an iterative process to develop queries for 7 adverse events, including those used by the Agency for Healthcare Research and Quality as patient safety indicators. We calculated positive predictive values of the 7 queries and performed an error analysis to detect reasons for false-positive cases of pulmonary embolism, deep vein thrombosis, and urinary tract infection.

RESULTS

Overall, 397,233 records were collected (34,805 discharge summaries, 292,593 emergency department notes, and 69,835 operation reports). Positive predictive values were higher for postoperative wound dehiscence (83.83%) and urinary tract infection (73.07%), whereas they were lower for deep vein thrombosis (5.37%), pulmonary embolism (13.63%), and postoperative sepsis (12.28%). The most common reasons for false positives were reporting of past events (42.25%), negations (22.80%), and conditions suspected by physicians but not confirmed by a diagnostic test (11.25%).

CONCLUSIONS

The results of our study demonstrated the feasibility of using an automated approach to detect multiple adverse events in several data sources. More sophisticated techniques, such as natural language processing, should be tested to evaluate the feasibility of using text mining as a routine method for monitoring adverse events in hospitals.

Natural language processing for the surveillance of postoperative venous thromboembolism

BACKGROUND

The objective of the study was to develop a portal natural language processing approach to aid in the identification of postoperative venous thromboembolism events from free-text clinical notes.

METHODS

We abstracted clinical notes from 25,494 operative events from 2 independent health care systems. A venous thromboembolism detected as part of the American College of Surgeons National Surgical Quality Improvement Program was used as the reference standard. A natural language processing engine, easy clinical information extractor-pulmonary embolism/deep vein thrombosis (EasyCIE-PEDVT), was trained to detect pulmonary embolism and deep vein thrombosis from clinical notes. International Classification of Diseases discharge diagnosis codes for venous thromboembolism were used as baseline comparators. The classification performance of EasyCIE-PEDVT was compared with International Classification of Diseases codes using sensitivity, specificity, area under the receiver operating characteristic curve, using an internal and external validation cohort.

RESULTS

To detect pulmonary embolism, EasyCIE-PEDVT had a sensitivity of 0.714 and 0.815 in internal and external validation, respectively. To detect deep vein thrombosis, EasyCIE-PEDVT had a sensitivity of 0.846 and 0.849 in internal and external validation, respectively. EasyCIE-PEDVT had significantly higher discrimination for deep vein thrombosis compared with International Classification of Diseases codes in internal validation (area under the receiver operating characteristic curve: 0.920 vs 0.761; P < .001) and external validation (area under the receiver operating characteristic curve: 0.921 vs 0.794; P < .001). There was no significant difference in the discrimination for pulmonary embolism between EasyCIE-PEDVT and International Classification of Diseases codes.

CONCLUSION

Accurate surveillance of postoperative venous thromboembolism may be achieved using natural language processing on clinical notes in 2 independent health care systems. These findings suggest natural language processing may augment manual chart abstraction for large registries such as National Surgical Quality Improvement Program.

Computerized Adjudication of Coronary Heart Disease Events Using the Electronic Medical Record in HIV Clinical Research: Possibilities and Challenges Ahead

This pilot study assessed feasibility of computer-assisted electronic medical record (EMR) abstraction to ascertain coronary heart disease (CHD) event hospitalizations. We included a sample of 87 hospitalization records from participants the University of North Carolina (UNC) site of the Women's Interagency HIV Study (WIHS) and UNC Center for AIDS Research (CFAR) HIV Clinical Cohort who were hospitalized within UNC Healthcare System from July 2004 to July 2015. We compared a computer algorithm utilizing diagnosis/procedure codes, medications, and cardiac enzyme levels to adjudicate CHD events [myocardial infarction (MI)/coronary revascularization] from the EMR to standardized manual chart adjudication. Of 87 hospitalizations, 42 were classified as definite, 25 probable, and 20 non-CHD events by manual chart adjudication. A computer algorithm requiring presence of ≥1 CHD-related International Classification of Diseases, 9th Revision (ICD-9)/Current Procedural Terminology (CPT) code correctly identified 24 of 42 definite (57%), 29 of 67 probable/definite CHD (43%), and 95% of non-CHD events; additionally requiring clinically defined cardiac enzyme levels or administration of MI-related medications correctly identified 55%, 42%, and 95% of such events, respectively. Requiring any one of the ICD-9/CPT or cardiac enzyme criteria correctly identified 98% of definite, 97% of probable/definite CHD, and 85% of non-CHD events. Challenges included difficulty matching hospitalization dates, incomplete diagnosis code data, and multiple field names/locations of laboratory/medication data. Computer algorithms comprising only ICD-9/CPT codes failed to identify a sizable proportion of CHD events. Using a less restrictive algorithm yielded fewer missed events but increased the false-positive rate. Despite potential benefits of EMR-based research, there remain several challenges to fully computerized adjudication of CHD events.

Automated screening of natural language in electronic health records for the diagnosis septic shock is feasible and outperforms an approach based on explicit administrative codes

PURPOSE

Identification of patients for epidemiologic research through administrative coding has important limitations. We investigated the feasibility of a search based on natural language processing (NLP) on the text sections of electronic health records for identification of patients with septic shock.

MATERIALS AND METHODS

Results of an explicit search strategy (using explicit concept retrieval) and a combined search strategy (using both explicit and implicit concept retrieval) were compared to hospital ICD-9 based administrative coding and to our department's own prospectively compiled infection database.

RESULTS

Of 8911 patients admitted to the medical or surgical ICU, 1023 (11.5%) suffered from septic shock according to the combined search strategy. This was significantly more than those identified by the explicit strategy (518, 5.8%), by hospital administrative coding (549, 5.8%) or by our own prospectively compiled database (609, 6.8%) (p < .001). Sensitivity and specificity of the automated combined search strategy were 72.7% (95%CI 69.0%-76.2%) and 93.0% (95%CI 92.4%-93.6%), compared to 56.0% (95%CI 52.0%-60.0%) and 97.5% (95%CI 97.1%-97.8%) for hospital administrative coding.

CONCLUSIONS

An automated search strategy based on a combination of explicit and implicit concept retrieval is feasible to screen electronic health records for septic shock and outperforms an administrative coding based explicit approach.

Anterior Versus Transforaminal Lumbar Interbody Fusion: Perioperative Risk Factors and 30-Day Outcomes

Background

Operative management of lower back pain often necessitates anterior lumbar interbody fusion (ALIF) or transforaminal lumbar interbody fusion (TLIF). Specific pathoanatomic advantages and indications exist for both approaches, and few studies to date have characterized comparative early outcomes.

Methods

Adult patients undergoing elective ALIF or TLIF operations were abstracted from the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) years 2011-2014. Univariate analyses were performed by surgery cohort for each outcome and adjusted for demographic/clinical variables (age ≥ 65, sex, race, body mass index, American Society of Anesthesiologists physical classification score, functional status, inpatient/outpatient status, smoking, hypertension, Charlson Comorbidity Index) using multivariable regression. Means, standard errors, mean differences (B), odds ratios (ORs), and associated 95% confidence intervals (CIs) are reported. Significance was assessed at P < .05.

Results

Of 8263 subjects (ALIF: 4325, TLIF: 3938), ALIF subjects were younger, less obese, less physically impaired, and had significantly lower rates of hypertension, diabetes, coagulopathy, and previous cardiac surgery. On multivariable analysis, ALIF associated with shorter operative time (B = -11.80 minutes, 95% CI [-16.48, -7.12]; P < .001). Transforaminal lumbar interbody fusion was associated with increased incidence of urinary tract infections (UTIs; OR = 1.57, 95% CI [1.10, 2.26]; P = .013) and of blood transfusions (OR = 1.19, 95% CI [1.04, 1.37]; P = .012). Multivariate analysis also demonstrated TLIF associated with shorter hospital length of stay (B = -0.27 days, 95% CI [-0.54, -0.01]; P = .041), and fewer cases of pneumonia (OR = 0.55, 95% CI [0.32, 0.94]; P = .029) and prolonged ventilator dependency (OR = 0.33, 95% CI [0.12, 0.84]; P = .021).

Conclusions

Comparatively, ALIF patients experienced decreased operative time and decreased incidence of postoperative UTIs and blood transfusions. Anterior lumbar interbody fusion patients were more likely to suffer postoperative pulmonary complications and longer hospital stays. Our data support the notion that both anterior and transforaminal surgical approaches perform comparably in context of 30-day perioperative outcomes.

Automated Extraction of VTE Events From Narrative Radiology Reports in Electronic Health Records: A Validation Study

BACKGROUND

Surveillance of venous thromboembolisms (VTEs) is necessary for improving patient safety in acute care hospitals, but current detection methods are inaccurate and inefficient. With the growing availability of clinical narratives in an electronic format, automated surveillance using natural language processing (NLP) techniques may represent a better method.

OBJECTIVE

We assessed the accuracy of using symbolic NLP for identifying the 2 clinical manifestations of VTE, deep vein thrombosis (DVT) and pulmonary embolism (PE), from narrative radiology reports.

METHODS

A random sample of 4000 narrative reports was selected among imaging studies that could diagnose DVT or PE, and that were performed between 2008 and 2012 in a university health network of 5 adult-care hospitals in Montreal (Canada). The reports were coded by clinical experts to identify positive and negative cases of DVT and PE, which served as the reference standard. Using data from the largest hospital (n=2788), 2 symbolic NLP classifiers were trained; one for DVT, the other for PE. The accuracy of these classifiers was tested on data from the other 4 hospitals (n=1212).

RESULTS

On manual review, 663 DVT-positive and 272 PE-positive reports were identified. In the testing dataset, the DVT classifier achieved 94% sensitivity (95% CI, 88%-97%), 96% specificity (95% CI, 94%-97%), and 73% positive predictive value (95% CI, 65%-80%), whereas the PE classifier achieved 94% sensitivity (95% CI, 89%-97%), 96% specificity (95% CI, 95%-97%), and 80% positive predictive value (95% CI, 73%-85%).

CONCLUSIONS

Symbolic NLP can accurately identify VTEs from narrative radiology reports. This method could facilitate VTE surveillance and the evaluation of preventive measures.

Risk factors for 30-day outcomes in elective anterior versus posterior cervical fusion: A matched cohort analysis

OBJECTIVE

Cervical spine fusion is the preferred treatment modality for a variety of degenerative and/or myelopathic disorders. Surgeons select between two approaches (anterior or posterior cervical fusion [ACF; PCF]) based on pathoanatomical features and spinal levels involved. Complications and outcome profiles between the approaches following elective surgery have not been systematically investigated.

METHODS

Adult patients undergoing elective ACF or PCF were extracted from the American College of Surgeons National Surgical Quality Improvement Program years 2011-2014. Five hundred twenty-eight patients (264 ACF and 264 PCF) were matched 1:1 by age, sex, functional status, vertebral levels operated, and the American Society of Anesthesiologists classification. Multivariable regression was performed by surgical approach for operation time, complications, hospital length of stay (HLOS), and discharge destination, controlling for body mass index and comorbidities. Mean differences (B), odds ratios (ORs), and 95% confidence intervals (CIs) are reported.

RESULTS

Compared to ACF, PCF was associated with increased odds of blood transfusions >1 unit (OR = 4.31, 95% CI [1.18-15.75]; P = 0.027) and failure to discharge to home (OR = 3.68 [2.17-6.25]; P < 0.001), and increased mean HLOS (B = 1.72 days [1.19-2.26]; P < 0.001). No differences in operation time, other complications, or reoperation rates were found by surgical approach.

CONCLUSIONS

In a matched cohort analysis by age, sex, functional and physical status, and vertebral levels, elective PCF is associated with increased HLOS and increased likelihood of failing to discharge to home compared to ACF without increased risk of 30-day complications. Increased blood transfusion volume is noted for patients undergoing PCF. Future prospective studies are warranted.

A novel use of the discrete templated notes within an electronic health record software to monitor resident supervision

Objective

Monitoring the supervision of residents can be a challenging task. We describe our experience with the implementation of a templated note system for documenting procedures with the aim of enabling automated, discrete, and standardized capture of documentation of supervision of residents performing floor-based procedures, with minimal extra effort from the residents.

Materials and methods

Procedural note templates were designed using the standard existing template within a commercial electronic health record software. Templates for common procedures were created such that residents could document every procedure performed outside of the formal procedural areas. Automated reports were generated and letters were sent to noncompliers.

Results

A total of 27 045 inpatient non-formal procedural area procedures were recorded from August 2012 to June 2014. Compliance with NoteWriter template usage averaged 86% in the first year and increased to 94.6% in the second year ( P  = .0055).

Initially, only 12.5% of residents documented supervision of any form. By the end of the first year, this was above 80%, with the gains maintained into the second year and beyond. Direct supervision was documented to have occurred where required in 62.8% in the first year and increased to 99.8% in the second year ( P  = .0001) after the addition of hard stops. Notification of attendings prior to procedures was documented 100% of the time by September 2013. Letters sent to errant residents decreased from 3.6 to 0.83 per 100 residents per week.

Conclusion

The templated procedure note system with hard stops and integrated reporting can successfully be used to improve monitoring of resident supervision. This has potential impact on resident education and patient safety.

MyPOD: an EMR-Based Tool that Facilitates Quality Improvement and Maintenance of Certification

Maintenance of Certification (MOC) was designed to assess physician competencies including operative case volume and outcomes. This information, if collected consistently and systematically, can be used to facilitate quality improvement. Information automatically extracted from the electronic medical record (EMR) can be used as a prompt to compile these data. We developed an EMR-based program called MyPOD (My Personal Outcomes Data) to track surgical outcomes at our institution. We compared occurrences reported in the first 18 months to those captured in the American College of Surgeons National Surgical Quality Improvement Program-Pediatric (ACS NSQIP-P) over the same time period. During the first 18 months of using MyPOD, 691 cases were captured in both MyPOD and NSQIP-P. There were 48 cases with occurrences in NSQIP-P (6.9% occurrence rate). MyPOD captured 33% of the occurrences and 83% of the deaths reported in NSQIP-P. Use of the MyPOD program helped to identify series of complications and facilitated systematic change to improve outcomes. MyPOD provides comparative data that is essential in performance evaluation and facilitates quality improvement in surgery. This program and similar EMR-driven tools are becoming essential components of the MOC process. Our initial review has revealed opportunities for improvement in self-reporting which we can continue to measure by comparison to NSQIP-P. In addition, it has identified systems issues that have led to hospital-wide improvements.

Obesity Is an Independent Risk Factor of Early Complications After Revision Spine Surgery

STUDY DESIGN

Retrospective cohort analysis of risk factors in revision spine surgery using a prospectively collected database.

OBJECTIVE

To examine the risk of developing early (30-day) complications across obesity level after adjusting for comorbidities in patients undergoing revision spine surgery.

SUMMARY OF BACKGROUND DATA

Prior studies suggest obesity influences early complications after primary surgery. The association between obesity and early complications after revision surgery remains to be characterized.

METHODS

Data were abstracted from the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) database from 2005 to 2012. Adult Caucasian patients undergoing removal/revision of instrumentation or exploration of fusion were included. Patients were categorized by WHO body mass index (BMI, kg/m): Non-Obese (18.5-29.9), Obese Class I (30-34.9), and Obese Class II/III (≥35). Univariate regression was performed to assess the predictive value of obesity level and baseline risk factors in the presence of at least one early complication, and significant predictors were entered into the multivariable model.

RESULTS

Of 2538 patients, 57.6% were nonobese, 23% Obese Class I, and 19.4% Obese Class II/III. Obesity was associated with diabetes, hypertension, respiratory disease, and American Society of Anesthesiologists (ASA) score of 3-4 (all P < 0.001). BMI group (P = 0.01), older age (P = 0.008), functional dependence (P < 0.001), ASA 3-4 (P = 0.008), bleeding disorder (P = 0.04), and diabetes (P = 0.016) were identified as univariate predictors for early complications. In the multivariable model, higher BMI (P = 0.04), older age (P = 0.014), and functional dependence (P < 0.001) remained significant predictors for early complications. Notably, patients who were Obese Class II/III (OR 1.66, 95% CI [1.12-2.45]), age ≥75 (OR 1.83, [1.20-2.81]), and functionally dependent (OR 3.02 [1.85-4.94]) had significantly higher risk compared with their reference groups.

CONCLUSION

Obesity is an independent risk factor for early complications after revision spine surgery. Although obesity may not contraindicate revision surgery, its status as a modifiable risk factor warrants disclosure and preoperative counseling to optimize outcomes.

LEVEL OF EVIDENCE

3.

Improving quality on the pediatric surgery service: Missed opportunities and making it happen

In surgery, quality improvement efforts have evolved from the traditional case-by-case review typical for morbidity and mortality conferences to more accurate and comprehensive data collection accomplished through participation in national registries such as the National Surgical Quality Improvement Program. Gaining administrative support to participate in these kinds of initiatives and commitment of the faculty and staff to make change in a data-driven manner rather than as a reaction to individual events can be a challenge. This article guides the reader through the process of interacting with administrative leadership to gain support for evidence-based quality improvement endeavors. General principles that are discussed include stakeholder engagement, taking advantage of preexisting resources, and the sharing of data in order to shape QI efforts and demonstrate their effectiveness.

Diagnosis and treatment of acute sinusitis in the primary care setting: A retrospective cohort

OBJECTIVES AND HYPOTHESIS

Our objectives were to characterize the quality of acute sinusitis care and to identify nonclinical factors associated with antibiotic use for acute sinusitis. We hypothesized that we would identify provider-level factors associated with antibiotic use.

STUDY DESIGN

Retrospective cohort at a single academic institution.

METHODS

We developed and clinically annotated an administrative dataset of adult patients diagnosed with acute sinusitis between January 1, 2005, and December 31, 2006. We used identify factors associated with receipt of antibiotics.

RESULTS

We find that 66.0% of patients with mild symptoms of short duration are given antibiotics, and that nonclinical factors, including the individual provider, the provider's specialty, and the presence of a medical trainee, significantly influence antibiotic use. Relative to internal medicine providers, family medicine providers use fewer antibiotics, and emergency medicine providers use more antibiotics for acute sinusitis.

CONCLUSIONS

Antibiotics continue to be overused for patients with mild acute sinusitis of short duration. Nonclinical characteristics, including the individual provider, the provider's specialty, and the presence of a medical trainee, significantly influence use of antibiotics for acute sinusitis.

LEVEL OF EVIDENCE

4.

Supporting information retrieval from electronic health records: A report of University of Michigan's nine-year experience in developing and using the Electronic Medical Record Search Engine (EMERSE)

OBJECTIVE

This paper describes the University of Michigan's nine-year experience in developing and using a full-text search engine designed to facilitate information retrieval (IR) from narrative documents stored in electronic health records (EHRs). The system, called the Electronic Medical Record Search Engine (EMERSE), functions similar to Google but is equipped with special functionalities for handling challenges unique to retrieving information from medical text.

MATERIALS AND METHODS

Key features that distinguish EMERSE from general-purpose search engines are discussed, with an emphasis on functions crucial to (1) improving medical IR performance and (2) assuring search quality and results consistency regardless of users' medical background, stage of training, or level of technical expertise.

RESULTS

Since its initial deployment, EMERSE has been enthusiastically embraced by clinicians, administrators, and clinical and translational researchers. To date, the system has been used in supporting more than 750 research projects yielding 80 peer-reviewed publications. In several evaluation studies, EMERSE demonstrated very high levels of sensitivity and specificity in addition to greatly improved chart review efficiency.

DISCUSSION

Increased availability of electronic data in healthcare does not automatically warrant increased availability of information. The success of EMERSE at our institution illustrates that free-text EHR search engines can be a valuable tool to help practitioners and researchers retrieve information from EHRs more effectively and efficiently, enabling critical tasks such as patient case synthesis and research data abstraction.

CONCLUSION

EMERSE, available free of charge for academic use, represents a state-of-the-art medical IR tool with proven effectiveness and user acceptance.

Unbiased identification of patients with disorders of sex development

Disorders of sex development (DSD) represent a collection of rare diseases that generate substantial controversy regarding best practices for diagnosis and treatment. A significant barrier preventing a better understanding of how patients with these conditions should be evaluated and treated, especially from a psychological standpoint, is the lack of systematic and standardized approaches to identify cases for study inclusion. Common approaches include "hand-picked" subjects already known to the practice, which could introduce bias. We implemented an informatics-based approach to identify patients with DSD from electronic health records (EHRs) at three large, academic children's hospitals. The informatics approach involved comprehensively searching EHRs at each hospital using a combination of structured billing codes as an initial filtering strategy followed by keywords applied to the free text clinical documentation. The informatics approach was implemented to replicate the functionality of an EHR search engine (EMERSE) available at one of the hospitals. At the two hospitals that did not have EMERSE, we compared case ascertainment using the informatics method to traditional approaches employed for identifying subjects. Potential cases identified using all approaches were manually reviewed by experts in DSD to verify eligibility criteria. At the two institutions where both the informatics and traditional approaches were applied, the informatics approach identified substantially higher numbers of potential study subjects. The traditional approaches yielded 14 and 28 patients with DSD, respectively; the informatics approach yielded 226 and 77 patients, respectively. The informatics approach missed only a few cases that the traditional approaches identified, largely because those cases were known to the study team, but patient data were not in the particular children's hospital EHR. The use of informatics approaches to search electronic documentation can result in substantially larger numbers of subjects identified for studies of rare diseases such as DSD, and these approaches can be applied across hospitals.

Development and testing of tools to detect ambulatory surgical adverse events

OBJECTIVES

Numerous health-care systems in the United States, including the Veterans Health Administration (VA), use the National Surgical Quality Improvement Program (NSQIP) to detect surgical adverse events (AEs). VASQIP sampling methodology excludes many routine ambulatory surgeries from review. Triggers, algorithms derived from clinical logic to flag cases where AEs have most likely occurred, could complement VASQIP by detecting a higher yield of ambulatory surgeries with a true surgical AE.

METHODS

We developed and tested a set of ambulatory surgical AE trigger algorithms using a sample of fiscal year 2008 ambulatory surgeries from the VA Boston Healthcare System. We used VA Boston VASQIP-assessed cases to refine triggers and VASQIP-excluded cases to test how many trigger-flagged surgeries had a nurse chart review-detected surgical AE. Chart review was performed using the VA electronic medical record. We calculated the ratio of cases with a true surgical AE over flagged cases (i.e., the positive predictive value [PPV]), and the 95% confidence interval for each trigger.

RESULTS

Compared with the VASQIP rate (9 AEs, or 2.8%, of the 322 charts assessed), nurse chart review of the 198 trigger-flagged surgeries yielded more cases with at least 1 AE (47 surgeries with an AE, or 6.0%, of the 782 VASQIP-excluded ambulatory surgeries). Individual trigger PPVs ranged from 12.4% to 58.3%.

CONCLUSIONS

In comparison with VASQIP, our set of triggers identified a higher rate of surgeries with AEs in fewer chart-reviewed cases. Because our results are based on a relatively small sample, further research is necessary to confirm these findings.

Towards prevention of acute syndromes: electronic identification of at-risk patients during hospital admission

BACKGROUND

Identifying patients at risk for acute respiratory distress syndrome (ARDS) before their admission to intensive care is crucial to prevention and treatment. The objective of this study is to determine the performance of an automated algorithm for identifying selected ARDS predisposing conditions at the time of hospital admission.

METHODS

This secondary analysis of a prospective cohort study included 3,005 patients admitted to hospital between January 1 and December 31, 2010. The automated algorithm for five ARDS predisposing conditions (sepsis, pneumonia, aspiration, acute pancreatitis, and shock) was developed through a series of queries applied to institutional electronic medical record databases. The automated algorithm was derived and refined in a derivation cohort of 1,562 patients and subsequently validated in an independent cohort of 1,443 patients. The sensitivity, specificity, and positive and negative predictive values of an automated algorithm to identify ARDS risk factors were compared with another two independent data extraction strategies, including manual data extraction and ICD-9 code search. The reference standard was defined as the agreement between the ICD-9 code, automated and manual data extraction.

RESULTS

Compared to the reference standard, the automated algorithm had higher sensitivity than manual data extraction for identifying a case of sepsis (95% vs. 56%), aspiration (63% vs. 42%), acute pancreatitis (100% vs. 70%), pneumonia (93% vs. 62%) and shock (77% vs. 41%) with similar specificity except for sepsis and pneumonia (90% vs. 98% for sepsis and 95% vs. 99% for pneumonia). The PPV for identifying these five acute conditions using the automated algorithm ranged from 65% for pneumonia to 91 % for acute pancreatitis, whereas the NPV for the automated algorithm ranged from 99% to 100%.

CONCLUSION

A rule-based electronic data extraction can reliably and accurately identify patients at risk of ARDS at the time of hospital admission.

Acute pain medicine in anesthesiology

The American Academy of Pain Medicine and the American Society for Regional Anesthesia have recently focused on the evolving practice of acute pain medicine. There is increasing recognition that the scope and practice of acute pain therapies must extend beyond the subacute pain phase to include pre-pain and pre-intervention risk stratification, resident and fellow education in regional anesthesia and multimodal analgesia, as well as a deeper understanding of the pathophysiologic mechanisms that are integral to the variability observed among individual responses to nociception. Acute pain medicine is also being established as a vital component of successful systems-level acute pain management programs, inpatient cost containment, and patient satisfaction scores. In this review, we discuss the evolution and practice of acute pain medicine and we aim to facilitate further discussion on the evolution and advancement of this field as a subspecialty of anesthesiology.

Detecting adverse events in surgery: comparing events detected by the Veterans Health Administration Surgical Quality Improvement Program and the Patient Safety Indicators

BACKGROUND

The Patient Safety Indicators (PSIs) use administrative data to screen for select adverse events (AEs). In this study, VA Surgical Quality Improvement Program (VASQIP) chart review data were used as the gold standard to measure the criterion validity of 5 surgical PSIs. Independent chart review was also used to determine reasons for PSI errors.

METHODS

The sensitivity, specificity, and positive predictive value of PSI software version 4.1a were calculated among Veterans Health Administration hospitalizations (2003-2007) reviewed by VASQIP (n = 268,771). Nurses re-reviewed a sample of hospitalizations for which PSI and VASQIP AE detection disagreed.

RESULTS

Sensitivities ranged from 31% to 68%, specificities from 99.1% to 99.8%, and positive predictive values from 31% to 72%. Reviewers found that coding errors accounted for some PSI-VASQIP disagreement; some disagreement was also the result of differences in AE definitions.

CONCLUSIONS

These results suggest that the PSIs have moderate criterion validity; however, some surgical PSIs detect different AEs than VASQIP. Future research should explore using both methods to evaluate surgical quality.

Health care in adults with Down syndrome: a longitudinal cohort study

BACKGROUND

Individuals with Down syndrome increasingly survive into adulthood, yet little is known about their healthcare patterns as adults. Our study sought to characterise patterns of health care among adults with Down syndrome based on whether they had fully transitioned to adult-oriented providers by their inception in this cohort.

METHODS

In this retrospective observational cohort study, healthcare utilisation and annualised patient charges were evaluated in patients with Down syndrome aged 18-45 years who received care in a single academic health centre from 2000 to 2008. Comparisons were made based on patients' provider mix (only adult-focused or 'mixed' child- and adult-focused providers).

RESULTS

The cohort included 205 patients with median index age = 28 years; 52% of these adult patients had incompletely transitioned to adult providers and received components of their care from child-focused providers. A higher proportion of these 'mixed' patients were seen exclusively by subspecialty providers (mixed = 81%, adult = 46%, P < 0.001), suggesting a need for higher intensity specialised services. Patients in the mixed provider group incurred higher annualised charges in analyses adjusted for age, mortality, total annualised encounters, and number of subspecialty disciplines accessed. These differences were most pronounced when stratified by whether patients were hospitalised during the study period (e.g., difference in adjusted means between mixed versus adult provider groups: $571 without hospitalisation, $19,061 with hospitalisation).

CONCLUSIONS

In this unique longitudinal cohort of over 200 adults aged 18-45 years with Down syndrome, over half demonstrated incomplete transition to adult care. Persistent use of child-focused care, often with a subspecialty emphasis, has implications for healthcare charges. Future studies must identify reasons for distinct care patterns, examine their relationship with clinical outcomes, and evaluate which provider types deliver the highest quality care for adults with Down syndrome and a wide variety of comorbidities.

Optimising the use of electronic health records to estimate the incidence of rheumatoid arthritis in primary care: what information is hidden in free text?

Background

Primary care databases are a major source of data for epidemiological and health services research. However, most studies are based on coded information, ignoring information stored in free text. Using the early presentation of rheumatoid arthritis (RA) as an exemplar, our objective was to estimate the extent of data hidden within free text, using a keyword search.

Methods

We examined the electronic health records (EHRs) of 6,387 patients from the UK, aged 30 years and older, with a first coded diagnosis of RA between 2005 and 2008. We listed indicators for RA which were present in coded format and ran keyword searches for similar information held in free text. The frequency of indicator code groups and keywords from one year before to 14 days after RA diagnosis were compared, and temporal relationships examined.

Results

One or more keyword for RA was found in the free text in 29% of patients prior to the RA diagnostic code. Keywords for inflammatory arthritis diagnoses were present for 14% of patients whereas only 11% had a diagnostic code. Codes for synovitis were found in 3% of patients, but keywords were identified in an additional 17%. In 13% of patients there was evidence of a positive rheumatoid factor test in text only, uncoded. No gender differences were found. Keywords generally occurred close in time to the coded diagnosis of rheumatoid arthritis. They were often found under codes indicating letters and communications.

Conclusions

Potential cases may be missed or wrongly dated when coded data alone are used to identify patients with RA, as diagnostic suspicions are frequently confined to text. The use of EHRs to create disease registers or assess quality of care will be misleading if free text information is not taken into account. Methods to facilitate the automated processing of text need to be developed and implemented.

Exploring the frontier of electronic health record surveillance: the case of postoperative complications

BACKGROUND

The aim of this study was to build electronic algorithms using a combination of structured data and natural language processing (NLP) of text notes for potential safety surveillance of 9 postoperative complications.

METHODS

Postoperative complications from 6 medical centers in the Southeastern United States were obtained from the Veterans Affairs Surgical Quality Improvement Program (VASQIP) registry. Development and test datasets were constructed using stratification by facility and date of procedure for patients with and without complications. Algorithms were developed from VASQIP outcome definitions using NLP-coded concepts, regular expressions, and structured data. The VASQIP nurse reviewer served as the reference standard for evaluating sensitivity and specificity. The algorithms were designed in the development and evaluated in the test dataset.

RESULTS

Sensitivity and specificity in the test set were 85% and 92% for acute renal failure, 80% and 93% for sepsis, 56% and 94% for deep vein thrombosis, 80% and 97% for pulmonary embolism, 88% and 89% for acute myocardial infarction, 88% and 92% for cardiac arrest, 80% and 90% for pneumonia, 95% and 80% for urinary tract infection, and 77% and 63% for wound infection, respectively. A third of the complications occurred outside of the hospital setting.

CONCLUSIONS

Computer algorithms on data extracted from the electronic health record produced respectable sensitivity and specificity across a large sample of patients seen in 6 different medical centers. This study demonstrates the utility of combining NLP with structured data for mining the information contained within the electronic health record.

Retrospective database research in pediatric cardiology and congenital heart surgery: an illustrative example of limitations and possible solutions

BACKGROUND

Secondary use of data, whether from clinical information systems or registries, for carrying out clinical research in rare diseases is a common practice but is fraught with potential errors. We sought to elucidate some of the limitations of database research and describe possible solutions to overcome these limitations.

METHODS

Using a disease model of a rare postsurgical outcome, we evaluated the ability of four different data sources to correctly identify patients who had that outcome both as individual databases and also when used in conjunction with each other. These results were compared with manual chart review.

RESULTS

The sensitivity of the various databases to pick up a rare and specific outcome was poor (9.9%-37%), while the specificities were fairly good (91%-96.7%). By combining the databases, the sensitivity was increased to as much as 56.8% without a large decrease in the specificity (85.2%-91.6%). The electronic medical record (EMR) search engine had the highest sensitivity (96.9%) and a high specificity (89.3%) with a very high negative predictive value (99.4%).

CONCLUSION

For rare and specific diseases or outcomes, a single data source search methodology can miss large numbers of patients and potentially bias study results. Combining overlapping databases can improve the ability to capture these rare diseases or outcomes. While chart review remains the most accurate way to obtain complete case capture, new tools like EMR search engines can facilitate the efficiency of this process without sacrificing search quality.

Computer-facilitated review of electronic medical records reliably identifies emergency department interventions in older adults

OBJECTIVES

An estimated 14% to 25% of all scientific studies in peer-reviewed emergency medicine (EM) journals are medical records reviews. The majority of the chart reviews in these studies are performed manually, a process that is both time-consuming and error-prone. Computer-based text search engines have the potential to enhance chart reviews of electronic emergency department (ED) medical records. The authors compared the efficiency and accuracy of a computer-facilitated medical record review of ED clinical records of geriatric patients with a traditional manual review of the same data and describe the process by which this computer-facilitated review was completed.

METHODS

Clinical data from consecutive ED patients age 65 years or older were collected retrospectively by manual and computer-facilitated medical record review. The frequency of three significant ED interventions in older adults was determined using each method. Performance characteristics of each search method, including sensitivity and positive predictive value, were determined, and the overall sensitivities of the two search methods were compared using McNemar's test.

RESULTS

For 665 patient visits, there were 49 (7.4%) Foley catheters placed, 36 (5.4%) sedative medications administered, and 15 (2.3%) patients who received positive pressure ventilation. The computer-facilitated review identified more of the targeted procedures (99 of 100, 99%), compared to manual review (74 of 100 procedures, 74%; p < 0.0001).

CONCLUSIONS

A practical, non-resource-intensive, computer-facilitated free-text medical record review was completed and was more efficient and accurate than manually reviewing ED records.

Primary care for adults with Down syndrome: adherence to preventive healthcare recommendations

BACKGROUND

Due to significant medical improvements, persons with Down syndrome now live well into adulthood. Consequently, primary care for adults with Down syndrome needs to incorporate routine care with screening for condition-specific comorbidities. This study seeks to evaluate the adherence of primary care physicians to age- and condition-specific preventive care in a cohort of adults with Down syndrome.

METHODS

In this retrospective observational cohort study, preventive screening was evaluated in patients with Down syndrome aged 18-45 years who received primary care in an academic medical centre from 2000 to 2008. Comparisons were made based on the field of patients' primary care providers (Family or Internal Medicine).

RESULTS

This cohort included 62 patients, median index age = 33 years. Forty per cent of patients received primary care by Family Physicians, with 60% seen by Internal Medicine practices. Patient demographics, comorbidities and overall screening patterns were similar between provider groups. Despite near universal screening for obesity and hypothyroidism, adherence to preventive care recommendations was otherwise inconsistent. Screening was 'moderate' (50-80%) for cardiac anomalies, reproductive health, dentition, and the combined measure of behaviour, psychological, or memory abnormalities. Less than 50% of patients were evaluated for obstructive sleep apnea, atlanto-axial instability, hearing loss or vision loss.

CONCLUSIONS

We observed inconsistent preventive care in adults with Down syndrome over this 8.5-year study. This is concerning, given that the adverse effects of many of these conditions can be ameliorated if discovered in a timely fashion. Further studies must evaluate the implications of screening practices and more timely identification of comorbidities on clinical outcomes.

Collaborative search in electronic health records

OBJECTIVE

A full-text search engine can be a useful tool for augmenting the reuse value of unstructured narrative data stored in electronic health records (EHR). A prominent barrier to the effective utilization of such tools originates from users' lack of search expertise and/or medical-domain knowledge. To mitigate the issue, the authors experimented with a 'collaborative search' feature through a homegrown EHR search engine that allows users to preserve their search knowledge and share it with others. This feature was inspired by the success of many social information-foraging techniques used on the web that leverage users' collective wisdom to improve the quality and efficiency of information retrieval.

DESIGN

The authors conducted an empirical evaluation study over a 4-year period. The user sample consisted of 451 academic researchers, medical practitioners, and hospital administrators. The data were analyzed using a social-network analysis to delineate the structure of the user collaboration networks that mediated the diffusion of knowledge of search.

RESULTS

The users embraced the concept with considerable enthusiasm. About half of the EHR searches processed by the system (0.44 million) were based on stored search knowledge; 0.16 million utilized shared knowledge made available by other users. The social-network analysis results also suggest that the user-collaboration networks engendered by the collaborative search feature played an instrumental role in enabling the transfer of search knowledge across people and domains.

CONCLUSION

Applying collaborative search, a social information-foraging technique popularly used on the web, may provide the potential to improve the quality and efficiency of information retrieval in healthcare.

Apples and oranges: comparison of ACS-NSQIP observed outcomes with premier's quality manager-predicted outcomes

The National Surgical Quality Improvement Program (NSQIP) is used by the American College of Surgeons to measure and report surgical quality and outcomes. Premier's Quality Manager (QM) generates expected outcomes from patient charts. The authors compared observed NSQIP morbidity and mortality outcomes with those predicted by QM. NSQIP data for 1919 patients were entered into QM. The discriminatory accuracy of the QM model was assessed using the C statistic (1.0 implies perfect discrimination, and 0.5 implies no discrimination). NSQIP and QM both identified 51 deaths (C statistic, 0.91). NSQIP identified 478 postoperative occurrences, whereas QM predicted 714 patients with at least 1 complication; 223 of these were subclassified as patients with at least 1 morbid complication (C statistic, 0.83). QM did not perform as well in predicting the observed NSQIP morbidities. Surgical leaders and hospital administrators must critically evaluate products before adopting programs designed to improve patient outcomes or making decisions regarding physician practice.

The accuracy of complications documented in a prospective complication registry

BACKGROUND

The objectives of this study were to evaluate the accuracy of a prospective complication registry for documenting complications and identify possible factors for non-registering.

METHODS

Five hundred randomly selected patients admitted at the Department of Surgery of St. Elisabeth Hospital Tilburg, The Netherlands, in the year 2005, were evaluated for incidence and type of complications by an examination of their medical records and compared with a prospective complication registry. The system was independently reviewed by two persons for missing complications. Patient files with missing complications in the registry were screened for factors possibly responsible for non-registering.

RESULTS

Two hundred thirteen complications were detected, 58 (27%) missing in the registry. There were 50 different types of complications documented. The number of events missing per category were: drug-related (50%, n = 4), organ dysfunction (44%, n = 14), infection-related (25%, n = 19), surgery/intervention-related (23%, n = 14), and hospital-provider errors (19%, n = 7). Not all clinically important complications were adequately documented (e.g., anastomotic leakage). The kappa score was 0.695, making the interrater reliability substantial.

CONCLUSION

The accuracy of registering complications is fairly acceptable compared to the ranges mentioned in literature. It is disappointing that clinically important events are missing in the registry. The inaccuracy could be explained by a great diversity of documented events, due to a broad definition, suggesting ignorance of the responsible team of which events to register.

Follow-up disparities after trauma: a real problem for outcomes research

INTRODUCTION

The objectives of this study were to (1) determine risk factors associated with failure to follow-up (FTF) after traumatic injury and (2) in those patients who do follow up, to determine if information within the electronic medical record (EMR) is an adequate data-collection tool for outcomes research.

METHODS

A 6-year retrospective analysis was conducted on all admitted trauma patients using data from the trauma registry, National Death Index, 2000 Census Data, and the EMR. Bivariate and logistic regression analyses identified risk factors for FTF. A subgroup analysis evaluated the utility of using the EMR to determine basic functional outcomes (Glasgow outcome scale, diet, ambulation, and employment status).

RESULTS

A total of 14,784 patients were discharged, and 61% had follow-up appointments. Lower income, higher poverty rates, and lower education were significantly (P<.05) associated with FTF. Logistic regression analysis (excluding census data) identified that older age, lower Injury Severity Score, less severe head injury, nonwhite race, blunt injury, death after discharge, zip code within 25 miles, and patients discharged to home independently predicted FTF after traumatic injury. A subgroup analysis of the EMR showed the inability to reliably determine functional outcomes.

CONCLUSIONS

There are several disparities related to follow-up after trauma. Furthermore, charting deficiencies, even with an EMR, highlight the weaknesses of data available for trauma outcomes research. Trauma process improvement programs could target patients at risk for not following up and use a structured electronic outpatient note.

Enhanced identification of eligibility for depression research using an electronic medical record search engine

PURPOSE

Electronic medical records (EMRs) have become part of daily practice for many physicians. Attempts have been made to apply electronic search engine technology to speed EMR review. This was a prospective, observational study to compare the speed and clinical accuracy of a medical record search engine vs. manual review of the EMR.

METHODS

Three raters reviewed 49 cases in the EMR to screen for eligibility in a depression study using the electronic medical record search engine (EMERSE). One week later raters received a scrambled set of the same patients including 9 distractor cases, and used manual EMR review to determine eligibility. For both methods, accuracy was assessed for the original 49 cases by comparison with a gold standard rater.

RESULTS

Use of EMERSE resulted in considerable time savings; chart reviews using EMERSE were significantly faster than traditional manual review (p=0.03). The percent agreement of raters with the gold standard (e.g. concurrent validity) using either EMERSE or manual review was not significantly different.

CONCLUSIONS

Using a search engine optimized for finding clinical information in the free-text sections of the EMR can provide significant time savings while preserving clinical accuracy. The major power of this search engine is not from a more advanced and sophisticated search algorithm, but rather from a user interface designed explicitly to help users search the entire medical record in a way that protects health information.