A real-world exploration into clinical outcomes of direct oral anticoagulant therapy in people with chronic kidney disease: a large hospital-based study

BACKGROUND

There is limited evidence to support definite clinical outcomes of direct oral anticoagulant (DOAC) therapy in chronic kidney disease (CKD). By identifying the important variables associated with clinical outcomes following DOAC administration in patients in different stages of CKD, this study aims to assess this evidence gap.

METHODS

An anonymised dataset comprising 97,413 patients receiving DOAC therapy in a tertiary health setting was systematically extracted from the multidimensional electronic health records and prepared for analysis. Machine learning classifiers were applied to the prepared dataset to select the important features which informed covariate selection in multivariate logistic regression analysis.

RESULTS

For both CKD and non-CKD DOAC users, features such as length of stay, treatment days, and age were ranked highest for relevance to adverse outcomes like death and stroke. Patients with Stage 3a CKD had significantly higher odds of ischaemic stroke (OR 2.45, 95% Cl: 2.10-2.86; p = 0.001) and lower odds of all-cause mortality (OR 0.87, 95% Cl: 0.79-0.95; p = 0.001) on apixaban therapy. In patients with CKD (Stage 5) receiving apixaban, the odds of death were significantly lowered (OR 0.28, 95% Cl: 0.14-0.58; p = 0.001), while the effect on ischaemic stroke was insignificant.

CONCLUSIONS

A positive effect of DOAC therapy was observed in advanced CKD. Key factors influencing clinical outcomes following DOAC administration in patients in different stages of CKD were identified. These are crucial for designing more advanced studies to explore safer and more effective DOAC therapy for the population.

Sensitive Data Detection with High-Throughput Machine Learning Models in Electrical Health Records

In the era of big data, there is an increasing need for healthcare providers, communities, and researchers to share data and collaborate to improve health outcomes, generate valuable insights, and advance research. The Health Insurance Portability and Accountability Act of 1996 (HIPAA) is a federal law designed to protect sensitive health information by defining regulations for protected health information (PHI). However, it does not provide efficient tools for detecting or removing PHI before data sharing. One of the challenges in this area of research is the heterogeneous nature of PHI fields in data across different parties. This variability makes rule-based sensitive variable identification systems that work on one database fail on another. To address this issue, our paper explores the use of machine learning algorithms to identify sensitive variables in structured data, thus facilitating the de-identification process. We made a key observation that the distributions of metadata of PHI fields and non-PHI fields are very different. Based on this novel finding, we engineered over 30 features from the metadata of the original features and used machine learning to build classification models to automatically identify PHI fields in structured Electronic Health Record (EHR) data. We trained the model on a variety of large EHR databases from different data sources and found that our algorithm achieves 99% accuracy when detecting PHI-related fields for unseen datasets. The implications of our study are significant and can benefit industries that handle sensitive data.

Clinical outcome prediction using observational supervision with electronic health records and audit logs

OBJECTIVE

Audit logs in electronic health record (EHR) systems capture interactions of providers with clinical data. We determine if machine learning (ML) models trained using audit logs in conjunction with clinical data ("observational supervision") outperform ML models trained using clinical data alone in clinical outcome prediction tasks, and whether they are more robust to temporal distribution shifts in the data.

MATERIALS AND METHODS

Using clinical and audit log data from Stanford Healthcare, we trained and evaluated various ML models including logistic regression, support vector machine (SVM) classifiers, neural networks, random forests, and gradient boosted machines (GBMs) on clinical EHR data, with and without audit logs for two clinical outcome prediction tasks: major adverse kidney events within 120 days of ICU admission (MAKE-120) in acute kidney injury (AKI) patients and 30-day readmission in acute stroke patients. We further tested the best performing models using patient data acquired during different time-intervals to evaluate the impact of temporal distribution shifts on model performance.

RESULTS

Performance generally improved for all models when trained with clinical EHR data and audit log data compared with those trained with only clinical EHR data, with GBMs tending to have the overall best performance. GBMs trained with clinical EHR data and audit logs outperformed GBMs trained without audit logs in both clinical outcome prediction tasks: AUROC 0.88 (95% CI: 0.85-0.91) vs. 0.79 (95% CI: 0.77-0.81), respectively, for MAKE-120 prediction in AKI patients, and AUROC 0.74 (95% CI: 0.71-0.77) vs. 0.63 (95% CI: 0.62-0.64), respectively, for 30-day readmission prediction in acute stroke patients. The performance of GBM models trained using audit log and clinical data degraded less in later time-intervals than models trained using only clinical data.

CONCLUSION

Observational supervision with audit logs improved the performance of ML models trained to predict important clinical outcomes in patients with AKI and acute stroke, and improved robustness to temporal distribution shifts.

Health outcomes associated with patterns of substance use disorders among patients with type 2 diabetes and hypertension: Electronic health record findings

AIMS

To identify substance use disorder (SUD) patterns and their association with T2DM health outcomes among patients with type 2 diabetes and hypertension.

METHODS

We used latent class analysis on electronic health records from the MetroHealth System (Cleveland, Ohio) to obtain the target SUD groups: i) only tobacco (TUD), ii) tobacco and alcohol (TAUD), and iii) tobacco, alcohol, and at least one more substance (PSUD). A matching program with Mahalanobis distance within propensity score calipers created the matched control groups: no SUD (NSUD) for TUD and TUD for the other two SUD groups. The numbers of participants for the target-control groups were 8009 (TUD), 1672 (TAUD), and 642 (PSUD).

RESULTS

TUD was significantly associated with T2DM complications. Compared to TUD, the TAUD group showed a significantly higher likelihood for all-cause mortality (adjusted odds ratio (aOR) = 1.46) but not for any of the T2DM complications. Compared to TUD, the PSUD group experienced a significantly higher risk for cerebrovascular accident (CVA) (aOR = 2.19), diabetic neuropathy (aOR = 1.76), myocardial infarction (MI) (aOR = 1.76), and all-cause mortality (aOR = 1.66).

CONCLUSIONS

The findings of increased risk associated with PSUDs may provide insights for better management of patients with T2DM and hypertension co-occurrence.

Development and clinical application of an electronic health record quality control system for pulmonary aspergillosis based on guidelines and natural language processing technology

Background

There are considerable differences in the diagnosis and treatment of pulmonary aspergillosis (PA) between specialized hospitals and primary hospitals or developed areas and underdeveloped areas in China. There is a lack of electronic systems that assist respiratory physicians in standardizing the diagnosis and treatment of PA.

Methods

We extracted 26 quality control points from the latest guidelines related to PA, and developed a PA quality control system of electronic health record (EHR) based on natural language processing (NLP) techniques. We obtained PA patient records in the Department of Respiratory Medicine of the First Affiliated Hospital of Guangzhou Medical University to verify the effectiveness of the system comparing with manually evaluation of respiratory experts.

Results

We successfully developed quality control system of PA; 699 PA medical records from EHR of the First Affiliated Hospital of Guangzhou Medical University between January 2015 and March 2020 were obtained and assessed by the system; 162 defects were found, which included 19 medical records with diagnostic defects, 76 medical records with examination defects, and 80 medical records with treatment defects; 200 medical records were sampled for validation, and found that the sensitivity and accuracy of quality control system for pulmonary aspergillosis (QCSA) were 0.99 and 0.96, F1 value was 0.85, and the recall rate was 0.77 compared with experts' evaluation.

Conclusions

Our system successfully uses medical guidelines and NLP technology to detect defects in the diagnosis and treatment of PA, which helps to improve the management quality of PA patients.

DQAgui: a graphical user interface for the MIRACUM data quality assessment tool

Background

With the growing impact of observational research studies, there is also a growing focus on data quality (DQ). As opposed to experimental study designs, observational research studies are performed using data mostly collected in a non-research context (secondary use). Depending on the number of data elements to be analyzed, DQ reports of data stored within research networks can grow very large. They might be cumbersome to read and important information could be overseen quickly. To address this issue, a DQ assessment (DQA) tool with a graphical user interface (GUI) was developed and provided as a web application.

Methods

The aim was to provide an easy-to-use interface for users without prior programming knowledge to carry out DQ checks and to present the results in a clearly structured way. This interface serves as a starting point for a more detailed investigation of possible DQ irregularities. A user-centered development process ensured the practical feasibility of the interactive GUI. The interface was implemented in the R programming language and aligned to Kahn et al.’s DQ categories conformance, completeness and plausibility.

Results

With DQAgui, an R package with a web-app frontend for DQ assessment was developed. The GUI allows users to perform DQ analyses of tabular data sets and to systematically evaluate the results. During the development of the GUI, additional features were implemented, such as analyzing a subset of the data by defining time periods and restricting the analyses to certain data elements.

Conclusions

As part of the MIRACUM project, DQAgui is now being used at ten German university hospitals for DQ assessment and to provide a central overview of the availability of important data elements in a datamap over 2 years. Future development efforts should focus on design optimization and include a usability evaluation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01961-z.

Systematic Mapping Study of AI/Machine Learning in Healthcare and Future Directions

This study attempts to categorise research conducted in the area of: use of machine learning in healthcare, using a systematic mapping study methodology. In our attempt, we reviewed literature from top journals, articles, and conference papers by using the keywords use of machine learning in healthcare. We queried Google Scholar, resulted in 1400 papers, and then categorised the results on the basis of the objective of the study, the methodology adopted, type of problem attempted and disease studied. As a result we were able to categorize study in five different categories namely, interpretable ML, evaluation of medical images, processing of EHR, security/privacy framework, and transfer learning. In the study we also found that most of the authors have studied cancer, and one of the least studied disease was epilepsy, evaluation of medical images is the most researched and a new field of research, Interpretable ML/Explainable AI, is gaining momentum. Our basic intent is to provide a fair idea to future researchers about the field and future directions.

A machine learning approach to predicting risk of myelodysplastic syndrome

BACKGROUND

Early myelodysplastic syndrome (MDS) diagnosis can allow physicians to provide early treatment, which may delay advancement of MDS and improve quality of life. However, MDS often goes unrecognized and is difficult to distinguish from other disorders. We developed a machine learning algorithm for the prediction of MDS one year prior to clinical diagnosis of the disease.

METHODS

Retrospective analysis was performed on 790,470 patients over the age of 45 seen in the United States between 2007 and 2020. A gradient boosted decision tree model (XGB) was built to predict MDS diagnosis using vital signs, lab results, and demographics from the prior two years of patient data. The XGB model was compared to logistic regression (LR) and artificial neural network (ANN) models. The models did not use blast percentage and cytogenetics information as inputs. Predictions were made one year prior to MDS diagnosis as determined by International Classification of Diseases (ICD) codes, 9th and 10th revisions. Performance was assessed with regard to area under the receiver operating characteristic curve (AUROC).

RESULTS

On a hold-out test set, the XGB model achieved an AUROC value of 0.87 for prediction of MDS one year prior to diagnosis, with a sensitivity of 0.79 and specificity of 0.80. The XGB model was compared against LR and ANN models, which achieved an AUROC of 0.838 and 0.832, respectively.

CONCLUSIONS

Machine learning may allow for early MDS diagnosis MDS and more appropriate treatment administration.

Pharmacogenomics research and its clinical implementation in Thailand: Lessons learned from the resource-limited settings

Several barriers present challenges to implementing pharmacogenomics into practice. This review will provide an overview of the current pharmacogenomics practices and research in Thailand, address the challenges and lessons learned from delivering clinical pharmacogenomic services in Thailand, emphasize the pharmacogenomics implementation issues that must be overcome, and identify current pharmacogenomic initiatives and plans to facilitate clinical implementation of pharmacogenomics in Thailand. Ever since the pharmacogenomics research began in 2004 in Thailand, a multitude of pharmacogenomics variants associated with drug responses have been identified in the Thai population, such as HLA-B∗15:02 for carbamazepine and oxcarbazepine, HLA-B∗58:01 for allopurinol, HLA-B∗13:01 for dapsone and cotrimoxazole, CYP2B6 variants for efavirenz, CYP2C9∗3 for phenytoin and warfarin, CYP3A5∗3 for tacrolimus, and UGT1A1∗6 and UGT1A1∗28 for irinotecan, etc. The future of pharmacogenomics guided therapy in clinical settings across Thailand appears promising because of the availability of evidence of clinical validity of the pharmacogenomics testing and support for reimbursement of pharmacogenomics testing.

Patient Data-Sharing for AI: Ethical Challenges, Catholic Solutions

Recent news of Catholic and secular healthcare systems sharing electronic health record (EHR) data with technology companies for the purposes of developing artificial intelligence (AI) applications has drawn attention to the ethical and social challenges of such collaborations, including threats to patient privacy and confidentiality, undermining of patient consent, and lack of corporate transparency. Although the United States Catholic Conference of Bishops' Ethical and Religious Directives for Health Care Services (ERDs) address collaborations between US Catholic healthcare providers and other entities, the ERDs do not adequately address the novel concerns seen in EHR data-sharing for AI development. Neither does the Health Insurance Portability and Accountability Act (HIPAA) privacy rule. This article describes ethical and social problems observed in recent patient data-sharing collaborations with AI companies and analyzes them in light of the guiding principles of the ERDs as well as the 2020 Rome Call to AI Ethics (RCAIE) document recently released by the Vatican. While both the ERDs and RCAIE guiding principles can inform future collaborations, we suggest that the next revision of the ERDs should consider addressing data-sharing and AI more directly.

Summary

Electronic health record data-sharing with artificial intelligence developers presents unique ethical and social challenges that can be addressed with updated United States Catholic Conference of Bishops' Ethical and Religious Directives and guidance from the Vatican's 2020 Rome Call to AI Ethics.

Non-parametric recurrent events analysis with BART and an application to the hospital admissions of patients with diabetes

Much of survival analysis is concerned with absorbing events, i.e., subjects can only experience a single event such as mortality. This article is focused on non-absorbing or recurrent events, i.e., subjects are capable of experiencing multiple events. Recurrent events have been studied by many; however, most rely on the restrictive assumptions of linearity and proportionality. We propose a new method for analyzing recurrent events with Bayesian Additive Regression Trees (BART) avoiding such restrictive assumptions. We explore this new method via a motivating example of hospital admissions for diabetes patients and simulated data sets.

Automated grouping of medical codes via multiview banded spectral clustering

OBJECTIVE

With its increasingly widespread adoption, electronic health records (EHR) have enabled phenotypic information extraction at an unprecedented granularity and scale. However, often a medical concept (e.g. diagnosis, prescription, symptom) is described in various synonyms across different EHR systems, hindering data integration for signal enhancement and complicating dimensionality reduction for knowledge discovery. Despite existing ontologies and hierarchies, tremendous human effort is needed for curation and maintenance - a process that is both unscalable and susceptible to subjective biases. This paper aims to develop a data-driven approach to automate grouping medical terms into clinically relevant concepts by combining multiple up-to-date data sources in an unbiased manner.

METHODS

We present a novel data-driven grouping approach - multi-view banded spectral clustering (mvBSC) combining summary data from multiple healthcare systems. The proposed method consists of a banding step that leverages the prior knowledge from the existing coding hierarchy, and a combining step that performs spectral clustering on an optimally weighted matrix.

RESULTS

We apply the proposed method to group ICD-9 and ICD-10-CM codes together by integrating data from two healthcare systems. We show grouping results and hierarchies for 13 representative disease categories. Individual grouping qualities were evaluated using normalized mutual information, adjusted Rand index, and F1-measure, and were found to consistently exhibit great similarity to the existing manual grouping counterpart. The resulting ICD groupings also enjoy comparable interpretability and are well aligned with the current ICD hierarchy.

CONCLUSION

The proposed approach, by systematically leveraging multiple data sources, is able to overcome bias while maximizing consensus to achieve generalizability. It has the advantage of being efficient, scalable, and adaptive to the evolving human knowledge reflected in the data, showing a significant step toward automating medical knowledge integration.

Dissecting genetic factors affecting phenylephrine infusion rates during anesthesia: a genome-wide association study employing EHR data

BACKGROUND

The alpha-adrenergic agonist phenylephrine is often used to treat hypotension during anesthesia. In clinical situations, low blood pressure may require prompt intervention by intravenous bolus or infusion. Differences in responsiveness to phenylephrine treatment are commonly observed in clinical practice. Candidate gene studies indicate genetic variants may contribute to this variable response.

METHODS

Pharmacological and physiological data were retrospectively extracted from routine clinical anesthetic records. Response to phenylephrine boluses could not be reliably assessed, so infusion rates were used for analysis. Unsupervised k-means clustering was conducted on clean data containing 4130 patients based on phenylephrine infusion rate and blood pressure parameters, to identify potential phenotypic subtypes. Genome-wide association studies (GWAS) were performed against average infusion rates in two cohorts: phase I (n = 1205) and phase II (n = 329). Top genetic variants identified from the meta-analysis were further examined to see if they could differentiate subgroups identified by k-means clustering.

RESULTS

Three subgroups of patients with different response to phenylephrine were clustered and characterized: resistant (high infusion rate yet low mean systolic blood pressure (SBP)), intermediate (low infusion rate and low SBP), and sensitive (low infusion rate with high SBP). Differences among clusters were tabulated to assess for possible confounding influences. Comorbidity hierarchical clustering showed the resistant group had a higher prevalence of confounding factors than the intermediate and sensitive groups although overall prevalence is below 6%. Three loci with P < 1 × 10^-6 were associated with phenylephrine infusion rate. Only rs11572377 with P = 6.09 × 10^-7, a 3'UTR variant of EDN2, encoding a secretory vasoconstricting peptide, could significantly differentiate resistant from sensitive groups (P = 0.015 and 0.018 for phase I and phase II) or resistant from pooled sensitive and intermediate groups (P = 0.047 and 0.018).

CONCLUSIONS

Retrospective analysis of electronic anesthetic records data coupled with the genetic data identified genetic variants contributing to variable sensitivity to phenylephrine infusion during anesthesia. Although the identified top gene, EDN2, has robust biological relevance to vasoconstriction by binding to endothelin type A (ET_A) receptors on arterial smooth muscle cells, further functional as well as replication studies are necessary to confirm this association.

Comparing the Trends of Electronic Health Record Adoption Among Hospitals of the United States and Japan

The goal of this study is to examine the trends of Electronic Health Record (EHR) adoption among hospitals in Japan compared to those in the United States. Japan's nationwide survey of hospitals was utilized to extract the EHR adoption rates among Japanese hospitals. Comparable datasets from the Healthcare Information and Management System Society (HIMSS) and the American Hospital Association (AHA) were utilized to extract EHR adoption rates among U.S. hospitals. The trends of EHR adoption were stratified and analyzed by hospital size and hospital ownership status. As of 2014, the U.S. hospitals had a wider adoption of 'basic with clinical notes' EHRs compared to Japan (45.6% vs. 27.3%), but large hospitals (400+ beds) in Japan have shown a similar adoption rate of EHR systems than those of U.S. (65.6% vs. 68.5%). Governmental hospitals tend to be more advanced in EHR adoption than non-profit hospitals in Japan (53.0% vs. 21.5%). Non-profit hospitals show the highest adoption rate of 'basic' EHR systems in the U.S. as of 2014 (63.3%). Using the 'certified' definition of EHRs, the EHR adoption rate was close to 96% among U.S. hospitals as of 2016; however, updated EHR adoption data from Japanese hospitals has yet to be collected and published. U.S. and Japan have considerably increased EHR adoption among hospitals; however, this analysis indicates different trends of EHR adoption among hospitals by size and ownership status in both countries. Learnings from government programs supporting EHR adoption in the U.S. and Japan can be helpful in planning useful strategies for future hospital-oriented health IT policies in other developed nations.

Embedding Health Literacy Tools in Patient EHR Portals to Facilitate Productive Patient Engagement

Many health care providers have opened their EHR systems to patients in order to increase information sharing and patient participation. Accessing to EHR has offered the promises of improving patient understanding, engagement, and outcomes. Although patients generally appreciate the access to their health records, currently, most EHR systems are used as data storage and communication tools and their potential for promoting productive patient engagement have not fully developed. There is a need to develop and incorporate effective health literacy tools into EHR patient portals, helping patients interpret their health data, understand their medical conditions and treatment plans, make informed decisions, and take proper actions. We will examine the challenges that patients face in using EHR portals, then provide two innovative health literacy solutions for facilitating productive patient engagement: (a) an embedded semantic medical search engine that provides reliable and contextualized health information support, and (b) an integrated AI voice chatbot that answers patients' questions and provides on-demand self-care advice. Other approaches that can add benefits to patients in the context of using EHR will also be described.

Association of the CAN score with the FRAIL scale in community dwelling older adults

BACKGROUND

Frailty is a state of vulnerability to stressors which results in higher morbidity, mortality and healthcare utilization. The FRAIL scale is used as a validated screening for frailty. The Care Assessment Need (CAN) score is automatically generated from electronic health record data using a statistical model that includes data elements similar to the deficit accumulation model for frailty and predicts risk for hospitalization and/or mortality.

AIM

To determine the correlation of the CAN score with the FRAIL scale.

METHODS

A cross-sectional study of 503 community-dwelling older adults. We compared the FRAIL scale with the CAN score.

RESULTS

The CAN score was significantly different between robust, prefrail and frail. Post hoc analysis revealed significant increases in scores from robust to prefrail and frail groups, in that order. The CAN score and FRAIL scale showed a correlation.

CONCLUSIONS

The CAN score show a moderate positive association with the FRAIL scale.

The use of Electronic Health Records to Support Population Health: A Systematic Review of the Literature

Electronic health records (EHRs) have emerged among health information technology as "meaningful use" to improve the quality and efficiency of healthcare, and health disparities in population health. In other instances, they have also shown lack of interoperability, functionality and many medical errors. With proper implementation and training, are electronic health records a viable source in managing population health? The primary objective of this systematic review is to assess the relationship of electronic health records' use on population health through the identification and analysis of facilitators and barriers to its adoption for this purpose. Authors searched Cumulative Index of Nursing and Allied Health Literature (CINAHL) and MEDLINE (PubMed), 10/02/2012-10/02/2017, core clinical/academic journals, MEDLINE full text, English only, human species and evaluated the articles that were germane to our research objective. Each article was analyzed by multiple reviewers. Group members recognized common facilitators and barriers associated with EHRs effect on population health. A final list of articles was selected by the group after three consensus meetings (n = 55). Among a total of 26 factors identified, 63% (147/232) of those were facilitators and 37% (85/232) barriers. About 70% of the facilitators consisted of productivity/efficiency in EHRs occurring 33 times, increased quality and data management each occurring 19 times, surveillance occurring 17 times, and preventative care occurring 15 times. About 70% of the barriers consisted of missing data occurring 24 times, no standards (interoperability) occurring 13 times, productivity loss occurring 12 times, and technology too complex occurring 10 times. The analysis identified more facilitators than barriers to the use of the EHR to support public health. Wider adoption of the EHR and more comprehensive standards for interoperability will only enhance the ability for the EHR to support this important area of surveillance and disease prevention. This review identifies more facilitators than barriers to using the EHR to support public health, which implies a certain level of usability and acceptance to use the EHR in this manner. The public-health industry should combine their efforts with the interoperability projects to make the EHR both fully adopted and fully interoperable. This will greatly increase the availability, accuracy, and comprehensiveness of data across the country, which will enhance benchmarking and disease surveillance/prevention capabilities.

Design of the New York City Macroscope: Innovations in Population Health Surveillance Using Electronic Health Records

Introduction:

Electronic health records (EHRs) have the potential to offer real-time, inexpensive standardized health data about chronic health conditions. Despite rapid expansion, EHR data evaluations for chronic disease surveillance have been limited. We present design and methods for the New York City (NYC) Macroscope, an EHR-based chronic disease surveillance system.

This methods report is the first in a three part series describing the development and validation of the NYC Macroscope. This report describes in detail the infrastructure underlying the NYC Macroscope; indicator definitions; design decisions that were made to maximize data quality; characteristics of the population sampled; completeness of data collected; and lessons learned from doing this work. The second report describes the methods used to evaluate the validity and robustness of NYC Macroscope prevalence estimates; presents validation results for estimates of obesity, smoking, depression and influenza vaccination; and discusses the implications of our findings for NYC and for other jurisdictions embarking on similar work. The third report applies the same validation methods to metabolic outcomes, including the prevalence, treatment and control of diabetes, hypertension and hyperlipidemia.

Methods:

We designed the NYC Macroscope for comparison to a local “gold standard,” the 2013–14 NYC Health and Nutrition Examination Survey, and the telephonic 2013 Community Health Survey. NYC Macroscope indicators covered prevalence, treatment, and control of diabetes, hypertension, and hyperlipidemia; and prevalence of influenza vaccination, obesity, depression and smoking. Indicators were stratified by age, sex, and neighborhood poverty, and weighted to the in-care NYC population and limited to primary care patients. Indicator queries were distributed to a virtual network of primary care practices; 392 practices and 716,076 adult patients were retained in the final sample.

Findings:

The NYC Macroscope covered 10% of primary care providers and 15% of all adult patients in NYC in 2013 (8–47% of patients by neighborhood). Data completeness varied by domain from 98% for blood pressure among patients with hypertension to 33% for depression screening.

Discussion:

Design and validation efforts undertaken by NYC are described here to provide one potential blueprint for leveraging EHRs for population health monitoring. To replicate a model like NYC Macroscope, jurisdictions should establish buy-in; build informatics capacity; use standard, simple case defnitions; establish documentation quality thresholds; restrict to primary care providers; and weight the sample to a target population.

Monitoring Prevalence, Treatment, and Control of Metabolic Conditions in New York City Adults Using 2013 Primary Care Electronic Health Records: A Surveillance Validation Study

Introduction:

Electronic health records (EHRs) can potentially extend chronic disease surveillance, but few EHR-based initiatives tracking population-based metrics have been validated for accuracy. We designed a new EHR-based population health surveillance system for New York City (NYC) known as NYC Macroscope. This report is the third in a 3-part series describing the development and validation of that system. The first report describes governance and technical infrastructure underlying the NYC Macroscope. The second report describes validation methods and presents validation results for estimates of obesity, smoking, depression and influenza vaccination. In this third paper we present validation findings for metabolic indicators (hypertension, hyperlipidemia, diabetes).

Methods:

We compared EHR-based estimates to those from a gold standard surveillance source - the 2013–2014 NYC Health and Nutrition Examination Survey (NYC HANES) - overall and stratified by sex and age group, using the two one-sided test of equivalence and other validation criteria.

Results:

EHR-based hypertension prevalence estimates were highly concordant with NYC HANES estimates. Diabetes prevalence estimates were highly concordant when measuring diagnosed diabetes but less so when incorporating laboratory results. Hypercholesterolemia prevalence estimates were less concordant overall. Measures to assess treatment and control of the 3 metabolic conditions performed poorly.

Discussion:

While indicator performance was variable, findings here confirm that a carefully constructed EHR-based surveillance system can generate prevalence estimates comparable to those from gold-standard examination surveys for certain metabolic conditions such as hypertension and diabetes.

Conclusions:

Standardized EHR metrics have potential utility for surveillance at lower annual costs than surveys, especially as representativeness of contributing clinical practices to EHR-based surveillance systems increases.