Monitoring Prevalence, Treatment, and Control of Metabolic Conditions in New York City Adults Using 2013 Primary Care Electronic Health Records: A Surveillance Validation Study

Identification of Hypertension in Electronic Health Records Through Computable Phenotype Development and Validation for Use in Public Health Surveillance: Retrospective Study

BACKGROUND

Electronic health record (EHR) systems are widely used in the United States to document care delivery and outcomes. Health information exchange (HIE) networks, which integrate EHR data from the various health care providers treating patients, are increasingly used to analyze population-level data. Existing methods for population health surveillance of essential hypertension by public health authorities may be complemented using EHR data from HIE networks to characterize disease burden at the community level.

OBJECTIVE

We aimed to derive and validate computable phenotypes (CPs) to estimate hypertension prevalence for population-based surveillance using an HIE network.

METHODS

Using existing data available from an HIE network, we developed 6 candidate CPs for essential (primary) hypertension in an adult population from a medium-sized Midwestern metropolitan area in the United States. A total of 2 independent clinician reviewers validated the phenotypes through a manual chart review of 150 randomly selected patient records. We assessed the precision of CPs by calculating sensitivity, specificity, positive predictive value (PPV), F1-score, and validity of chart reviews using prevalence-adjusted bias-adjusted κ. We further used the most balanced CP to estimate the prevalence of hypertension in the population.

RESULTS

Among a cohort of 548,232 adults, 6 CPs produced PPVs ranging from 71% (95% CI 64.3%-76.9%) to 95.7% (95% CI 84.9%-98.9%). The F1-score ranged from 0.40 to 0.91. The prevalence-adjusted bias-adjusted κ revealed a high percentage agreement of 0.88 for hypertension. Similarly, interrater agreement for individual phenotype determination demonstrated substantial agreement (range 0.70-0.88) for all 6 phenotypes examined. A phenotype based solely on diagnostic codes possessed reasonable performance (F1-score=0.63; PPV=95.1%) but was imbalanced with low sensitivity (47.6%). The most balanced phenotype (F1-score=0.91; PPV=83.5%) included diagnosis, blood pressure measurements, and medications and identified 210,764 (38.4%) individuals with hypertension during the study period (2014-2015).

CONCLUSIONS

We identified several high-performing phenotypes to identify essential hypertension prevalence for local public health surveillance using EHR data. Given the increasing availability of EHR systems in the United States and other nations, leveraging EHR data has the potential to enhance surveillance of chronic disease in health systems and communities. Yet given variability in performance, public health authorities will need to decide whether to seek optimal balance or declare a preference for algorithms that lean toward sensitivity or specificity to estimate population prevalence of disease.

Utilization of Electronic Health Records for Chronic Disease Surveillance: A Systematic Literature Review

This study reviews the current utilization of electronic health records (EHRs) for chronic disease surveillance, discusses approaches that are used in obtaining EHR-derived disease prevalence estimates, and identifies health indicators that have been studied using EHR-based surveillance methods. PubMed was searched for relevant keywords: (electronic health records [Title/Abstract] AND surveillance [Title/Abstract]) OR (electronic medical records [Title/Abstract] AND surveillance [Title/Abstract]). Articles were assessed based on detailed inclusion and exclusion criteria and organized by common themes, as per the PRISMA review protocol. The study period was limited to 2015-2021 due to the wider adoption of EHR in the U.S. only since 2015. The review included only US studies and only those that focused on chronic disease surveillance. 17 studies were included in the review. The most common approaches the review identified focused on validating EHR-derived estimates against those from traditional national surveys. The most studied conditions were diabetes, obesity, and hypertension. The majority of reviewed studies demonstrated comparable prevalence estimates with traditional population health surveillance surveys. The most common approach for the estimation of chronic disease conditions was to use small-area estimation by geographic patterns, neighborhoods, or census tracts. The use of EHR-based surveillance systems for public health purposes is feasible, and the population health estimates appear comparable to those obtained through traditional surveillance surveys. The application of EHRs for public health surveillance appears promising and could offer a real-time alternative to traditional surveillance methods. A timely assessment of population health at local and regional levels would ensure a more targeted allocation of public health and healthcare resources as well as more effective intervention and prevention initiatives.

State-level metabolic comorbidity prevalence and control among adults age 50-plus with diabetes: estimates from electronic health records and survey data in five states

BACKGROUND

Although treatment and control of diabetes can prevent complications and reduce morbidity, few data sources exist at the state level for surveillance of diabetes comorbidities and control. Surveys and electronic health records (EHRs) offer different strengths and weaknesses for surveillance of diabetes and major metabolic comorbidities. Data from self-report surveys suffer from cognitive and recall biases, and generally cannot be used for surveillance of undiagnosed cases. EHR data are becoming more readily available, but pose particular challenges for population estimation since patients are not randomly selected, not everyone has the relevant biomarker measurements, and those included tend to cluster geographically.

METHODS

We analyzed data from the National Health and Nutritional Examination Survey, the Health and Retirement Study, and EHR data from the DARTNet Institute to create state-level adjusted estimates of the prevalence and control of diabetes, and the prevalence and control of hypertension and high cholesterol in the diabetes population, age 50 and over for five states: Alabama, California, Florida, Louisiana, and Massachusetts.

RESULTS

The estimates from the two surveys generally aligned well. The EHR data were consistent with the surveys for many measures, but yielded consistently lower estimates of undiagnosed diabetes prevalence, and identified somewhat fewer comorbidities in most states.

CONCLUSIONS

Despite these limitations, EHRs may be a promising source for diabetes surveillance and assessment of control as the datasets are large and created during the routine delivery of health care.

TRIAL REGISTRATION

Not applicable.

A Case Study of Early-Onset Colorectal Cancer: Using Electronic Health Records to Support Public Health Surveillance on an Emerging Cancer Control Topic

Electronic health records (EHRs) are increasingly being used to support public health surveillance, including in cancer, where many population-based registries can now accept electronic case reporting. Using EHRs to supplement cancer registry data provides the opportunity to examine in more detail emerging issues in cancer control, such as the increasing incidence rates of early onset colorectal cancer (CRC). The purpose of this study was to evaluate the feasibility of a public health organization partnering with a health system to examine risk factors for early-onset CRC in a community cancer setting, and to further understand challenges with using EHRs to address emerging topics in cancer control. We conducted a mixed-methods evaluation using key informant interviews with public health practitioners, researchers, and registry staff to generate insights on how using EHRs and partnering with health systems can improve chronic disease surveillance and cancer control. A data quality assessment of variables representing risk factors for CRC and other clinical characteristics was conducted on all CRC patients diagnosed in 2016 at the participating cancer center. The quantitative assessment of the EHR data revealed that, while most chronic health conditions were well documented, around 25% of CRC patients were missing information on body mass index, alcohol, and tobacco use. Key informants offered ideas and ways to overcome challenges with using EHR data to support chronic disease surveillance. Their recommendations included the following activities: engaging EHR vendors in the development of standards, taking leadership roles on workgroups to address emerging technological issues, participating in pilot studies and task forces, and negotiating with EHR vendors so that clinical decision support tools built to support public health initiatives are freely available to all users of those EHRs. Although using EHR data to support public health efforts is not without its challenges, it soon could be an important part of chronic disease surveillance and cancer control.

Developing a Regional Distributed Data Network for Surveillance of Chronic Health Conditions: The Colorado Health Observation Regional Data Service

Electronic health records (EHRs) provide an alternative to traditional public health surveillance surveys and administrative data for measuring the prevalence and impact of chronic health conditions in populations. As the infrastructure for secondary use of EHR data improves, many stakeholders are poised to benefit from data partnerships for regional access to information. Electronic health records can be transformed into a common data model that facilitates data sharing across multiple organizations and allows data to be used for surveillance. The Colorado Health Observation Regional Data Service, a regional distributed data network, has assembled diverse data partnerships, flexible infrastructure, and transparent governance practices to better understand the health of communities through EHR-based, public health surveillance. This article describes attributes of regional distributed data networks using EHR data and the history and design of Colorado Health Observation Regional Data Service as an emerging public health surveillance tool for chronic health conditions. Colorado Health Observation Regional Data Service and our experience may serve as a model for other regions interested in similar surveillance efforts. While benefits from EHR-based surveillance are described, a number of technology, partnership, and value proposition challenges remain.

Prevalence estimation by joint use of big data and health survey: a demonstration study using electronic health records in New York city

Background

Electronic Health Records (EHR) has been increasingly used as a tool to monitor population health. However, subject-level errors in the records can yield biased estimates of health indicators. There is an urgent need for methods to estimate the prevalence of health indicators using large and real-time EHR while correcting the potential bias.

Methods

We demonstrate joint analyses of EHR and a smaller gold-standard health survey. We first adopted Mosteller’s method that pools two estimators, among which one is potentially biased. It only requires knowing the prevalence estimates from two data sources and their standard errors. Then, we adopted the method of Schenker et al., which uses multiple imputations of subject-level health outcomes that are missing for the subjects in EHR. This procedure requires information to link some subjects between two sources and modeling the mechanism of misclassification in EHR as well as modeling inclusion probabilities to both sources.

Results

In a simulation study, both estimators yielded negligible bias even when EHR was biased. They performed as well as health survey estimator when EHR bias was large and better than health survey estimator when EHR bias was moderate. It may be challenging to model the misclassification mechanism in real data for the subject-level imputation estimator. We illustrated the methods analyzing six health indicators from 2013 to 14 NYC HANES and the 2013 NYC Macroscope, and a study that linked some subjects in both data sources.

Conclusions

When a small gold-standard health survey exists, it can serve as a safeguard against potential bias in EHR through the joint analysis of the two sources.

Denominators Matter: Understanding Medical Encounter Frequency and Its Impact on Surveillance Estimates Using EHR Data

Background:

There is scant guidance for defining what denominator to use when estimating disease prevalence via electronic health record (EHR) data.

Objectives:

Describe the intervals between medical encounters to inform the selection of denominators for population-level disease rates, and evaluate the impact of different denominators on the prevalence of chronic conditions.

Methods:

We analyzed the EHRs of three practices in Massachusetts using the Electronic medical record Support for Public Health (ESP) system. We identified adult patients’ first medical encounter per year (2011–2016) and counted days to next encounter. We estimated the prevalence of asthma, hypertension, obesity, and smoking using different denominators in 2016: ≥1 encounter in the past one year or two years and ≥2 encounters in the past one year or two years.

Results:

In 2011–2016, 1,824,011 patients had 28,181,334 medical encounters. The median interval between encounters was 46, 56, and 66 days, depending on practice. Among patients with one visit in 2014, 82–84 percent had their next encounter within 1 year; 87–91 percent had their next encounter within two years. Increasing the encounter interval from one to two years increased the denominator by 23 percent. The prevalence of asthma, hypertension, and obesity increased with successively stricter denominators – e.g., the prevalence of obesity was 24.1 percent among those with ≥1 encounter in the past two years, 26.3 percent among those with ≥1 encounter in the last one year, and 28.5 percent among those with ≥2 encounters in the past one year.

Conclusions:

Prevalence estimates for chronic conditions can vary by >20 percent depending upon denominator. Understanding such differences will inform which denominator definition is best to be used for the need at hand.

Completeness of Electronic Dental Records in a Student Clinic: Retrospective Analysis

BACKGROUND

A well-designed, adequately documented, and properly maintained patient record is an important tool for quality assurance and care continuity. Good clinical documentation skills are supposed to be a fundamental part of dental student training.

OBJECTIVE

The goal of this study was to assess the completeness of electronic patient records in a student clinic.

METHODS

Completeness of patient records was assessed using comparative review of validated cases of alveolar osteitis treated between August 2011 and May 2017 in a student clinic at Columbia University College of Dental Medicine, New York, USA. Based on a literature review, population-based prevalence of nine most frequently mentioned symptoms, signs, and treatment procedures of alveolar osteitis was identified. Completeness of alveolar osteitis records was assessed by comparison of population-based prevalence and frequency of corresponding items in the student documentation. To obtain all alveolar osteitis cases, we ran a query on the electronic dental record, which included all cases with diagnostic code Z1820 or any variation of the phrases "dry socket" and "alveolar osteitis" in the notes. The resulting records were manually reviewed to definitively confirm alveolar osteitis and to extract all index items.

RESULTS

Overall, 296 definitive cases of alveolar osteitis were identified. Only 22% (64/296) of cases contained a diagnostic code. Comparison of the frequency of the nine index categories in the validated alveolar osteitis cases between the student clinic and the population showed the following results: severe pain: 94% (279/296) vs 100% (430/430); bare bone/missing blood clot: 27% (80/296) vs 74% (35/47) to 100% (329/329); malodor: 7% (22/296) vs 33%-50% (18/54); radiating pain to the ear: 8% (24/296) vs 56% (30/54); lymphadenopathy: 1% (3/296) vs 9% (5/54); inflammation: 14% (42/296) vs 50% (27/54); debris: 12% (36/296) vs 87% (47/54); alveolar osteitis site noted: 96% (283/296) vs 100% (430/430; accepted documentation requirement); and anesthesia during debridement: 77% (20/24) vs 100% (430/430; standard of anesthetization prior to debridement).

CONCLUSIONS

There was a significant discrepancy between the index category frequency in alveolar osteitis cases documented by dental students and in the population (reported in peer-reviewed literature). More attention to clinical documentation skills is warranted in dental student training.

Using Calibration to Reduce Measurement Error in Prevalence Estimates Based on Electronic Health Records

INTRODUCTION

Increasing adoption of electronic health record (EHR) systems by health care providers presents an opportunity for EHR-based population health surveillance. EHR data, however, may be subject to measurement error because of factors such as data entry errors and lack of documentation by physicians. We investigated the use of a calibration model to reduce bias of prevalence estimates from the New York City (NYC) Macroscope, an EHR-based surveillance system.

METHODS

We calibrated 6 health indicators to the 2013-2014 NYC Health and Nutrition Examination Survey (NYC HANES) data: hypertension, diabetes, smoking, obesity, influenza vaccination, and depression. We classified indicators into having low measurement error or high measurement error on the basis of whether the proportion of misclassification (ie, false-negative or false-positive cases) was greater than 15% in 190 reviewed charts. We compared bias (ie, absolute difference between NYC Macroscope estimates and NYC HANES estimates) before and after calibration.

RESULTS

The health indicators with low measurement error had the same bias after calibration as before calibration (diabetes, 2.5 percentage points; smoking, 2.5 percentage points; obesity, 3.5 percentage points; hypertension, 1.1 percentage points). For indicators with high measurement error, bias decreased from 10.8 to 2.5 percentage points for depression, and from 26.7 to 8.4 percentage points for influenza vaccination.

CONCLUSION

The calibration model has the potential to reduce bias of prevalence estimates from EHR-based surveillance systems for indicators with high measurement errors. Further research is warranted to assess the utility of the current calibration model for other EHR data and additional indicators.

Using the emergency department to investigate smoking in young adults

PURPOSE

Smoking in young adults identifies the population at risk for future tobacco-related disease. We investigated smoking in a young adult population and within high-risk groups using emergency department (ED) data in a metropolitan area.

METHODS

Using the electronic health record, we performed a retrospective study of smoking in adults aged 18-30 years presenting to the ED.

RESULTS

Smoking status was available for 55,777 subjects (90.9% of the total ED cohort); 60.8% were women, 55.0% were black, 35.3% were white, and 8.1% were Hispanic; 34.4% were uninsured. Most smokers used cigarettes (95.1%). Prevalence of current smoking was 21.7% for women and 42.5% for men. The electronic health record contains data about diagnosis and social history that can be used to investigate smoking status for high-risk populations. Smoking prevalence was highest for substance use disorder (58.0%), psychiatric illness (41.3%) and alcohol use (39.1%), and lowest for pregnancy (13.5%). In multivariable analyses, male gender, white race, lack of health insurance, alcohol use, and illicit drug use were independently associated with smoking. Smoking risk among alcohol and drug users varied by gender, race, and/or age.

CONCLUSIONS

The ED provides access to a large, demographically diverse population, and supports investigation of smoking risk in young adults.

Electronic health record case studies to advance environmental public health tracking

Data from traditional public health surveillance systems can have some limitations, e.g., timeliness, geographic level, and amount of data accessible. Electronic health records (EHRs) could present an opportunity to supplement current sources of routinely collected surveillance data. The National Environmental Public Health Tracking Program (Tracking Program) sought to explore the use of EHRs for advancing environmental public health surveillance practices. The Tracking Program funded four state/local health departments to obtain and pilot the use of EHR data to address several issues including the challenges and technical requirements for accessing EHR data, and the core data elements required to integrate EHR data within their departments' Tracking Programs. The results of these pilot projects highlighted the potential of EHR data for public health surveillance of rare diseases that may lack comprehensive registries, and surveillance of prevalent health conditions or risk factors for health outcomes at a finer geographic level. EHRs therefore, may have potential to supplement traditional sources of public health surveillance data.

Monitoring Depression Rates in an Urban Community: Use of Electronic Health Records

Objectives:

Depression is the most common mental health disorder and mediates outcomes for many chronic diseases. Ability to accurately identify and monitor this condition, at the local level, is often limited to estimates from national surveys. This study sought to compare and validate electronic health record (EHR)-based depression surveillance with multiple data sources for more granular demographic subgroup and subcounty measurements.

Design/Setting:

A survey compared data sources for the ability to provide subcounty (eg, census tract [CT]) depression prevalence estimates. Using 2011-2012 EHR data from 2 large health care providers, and American Community Survey data, depression rates were estimated by CT for Denver County, Colorado. Sociodemographic and geographic (residence) attributes were analyzed and described. Spatial analysis assessed for clusters of higher or lower depression prevalence.

Main Outcome Measure(s):

Depression prevalence estimates by CT.

Results:

National and local survey-based depression prevalence estimates ranged from 7% to 17% but were limited to county level. Electronic health record data provided subcounty depression prevalence estimates by sociodemographic and geographic groups (CT range: 5%-20%). Overall depression prevalence was 13%; rates were higher for women (16% vs men 9%), whites (16%), and increased with age and homeless patients (18%). Areas of higher and lower EHR-based, depression prevalence were identified.

Conclusions:

Electronic health record–based depression prevalence varied by CT, gender, race/ethnicity, age, and living status. Electronic health record–based surveillance complements traditional methods with greater timeliness and granularity. Validation through subcounty-level qualitative or survey approaches should assess accuracy and address concerns about EHR selection bias. Public health agencies should consider the opportunity and evaluate EHR system data as a surveillance tool to estimate subcounty chronic disease prevalence.

Generalizability of Indicators from the New York City Macroscope Electronic Health Record Surveillance System to Systems Based on Other EHR Platforms

Introduction:

The New York City (NYC) Macroscope is an electronic health record (EHR) surveillance system based on a distributed network of primary care records from the Hub Population Health System. In a previous 3-part series published in eGEMS, we reported the validity of health indicators from the NYC Macroscope; however, questions remained regarding their generalizability to other EHR surveillance systems.

Methods:

We abstracted primary care chart data from more than 20 EHR software systems for 142 participants of the 2013–14 NYC Health and Nutrition Examination Survey who did not contribute data to the NYC Macroscope. We then computed the sensitivity and specificity for indicators, comparing data abstracted from EHRs with survey data.

Results:

Obesity and diabetes indicators had moderate to high sensitivity (0.81–0.96) and high specificity (0.94–0.98). Smoking status and hypertension indicators had moderate sensitivity (0.78–0.90) and moderate to high specificity (0.88–0.98); sensitivity improved when the sample was restricted to records from providers who attested to Stage 1 Meaningful Use. Hyperlipidemia indicators had moderate sensitivity (≥0.72) and low specificity (≤0.59), with minimal changes when restricting to Stage 1 Meaningful Use.

Discussion:

Indicators for obesity and diabetes used in the NYC Macroscope can be adapted to other EHR surveillance systems with minimal validation. However, additional validation of smoking status and hypertension indicators is recommended and further development of hyperlipidemia indicators is needed.

Conclusion:

Our findings suggest that many of the EHR-based surveillance indicators developed and validated for the NYC Macroscope are generalizable for use in other EHR surveillance systems.

Academic and Government Partnerships to Address Diabetes in the USA: a Narrative Review

PURPOSE OF REVIEW

Multi-sector partnerships are broadly considered to be of value for diabetes prevention and management. The purpose of this article is to summarize academic and government collaborations focused on diabetes prevention and management.

RECENT FINDINGS

Using a narrative review approach, we identified 17 articles describing 10 academic and government partnerships for diabetes management and surveillance. Challenges and gaps in the literature include complexity of diabetes management vis a vis current healthcare infrastructure; a paucity of racial/ethnic diversity in translational efforts; and the time/effort needed to maintain strong relationships across partner institutions. Academic and government partnerships are of value for diabetes prevention and management activities. Acknowledgment that the key priorities of government programming are often costs and feasibility is critical for collaborations to be successful. Future translational efforts of diabetes prevention and management programs should focus on the following: (1) expansion of partnerships between academia and local health departments; (2) increased utilization of implementation science for enhanced and efficient implementation and dissemination; and (3) harnessing of technological advances for data analysis, patient communication, and report generation.

Comparing Prevalence Estimates From Population-Based Surveys to Inform Surveillance Using Electronic Health Records

INTRODUCTION

Electronic health record (EHR) systems provide an opportunity to use a novel data source for population health surveillance. Validation studies that compare prevalence estimates from EHRs and surveys most often use difference testing, which can, because of large sample sizes, lead to detection of significant differences that are not meaningful. We explored a novel application of the two one-sided t test (TOST) to assess the equivalence of prevalence estimates in 2 population-based surveys to inform margin selection for validating EHR-based surveillance prevalence estimates derived from large samples.

METHODS

We compared prevalence estimates of health indicators in the 2013 Community Health Survey (CHS) and the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES) by using TOST, a 2-tailed t test, and other goodness-of-fit measures.

RESULTS

A ±5 percentage-point equivalence margin for a TOST performed well for most health indicators. For health indicators with a prevalence estimate of less than 10% (extreme obesity [CHS, 3.5%; NYC HANES, 5.1%] and serious psychological distress [CHS, 5.2%; NYC HANES, 4.8%]), a ±2.5 percentage-point margin was more consistent with other goodness-of-fit measures than the larger percentage-point margins.

CONCLUSION

A TOST with a ±5 percentage-point margin was useful in establishing equivalence, but a ±2.5 percentage-point margin may be appropriate for health indicators with a prevalence estimate of less than 10%. Equivalence testing can guide future efforts to validate EHR data.

Innovations in Population Health Surveillance: Using Electronic Health Records for Chronic Disease Surveillance

With 87% of providers using electronic health records (EHRs) in the United States, EHRs have the potential to contribute to population health surveillance efforts. However, little is known about using EHR data outside syndromic surveillance and quality improvement. We created an EHR-based population health surveillance system called the New York City (NYC) Macroscope and assessed the validity of diabetes, hyperlipidemia, hypertension, smoking, obesity, depression, and influenza vaccination indicators. The NYC Macroscope uses aggregate data from a network of outpatient practices. We compared 2013 NYC Macroscope prevalence estimates with those from a population-based, in-person examination survey, the 2013-2014 NYC Health and Nutrition Examination Survey. NYC Macroscope diabetes, hypertension, smoking, and obesity prevalence indicators performed well, but depression and influenza vaccination estimates were substantially lower than were survey estimates. Ongoing validation will be important to monitor changes in validity over time as EHR networks mature and to assess new indicators. We discuss NYC's experience and how this project fits into the national context. Sharing lessons learned can help achieve the full potential of EHRs for population health surveillance.

Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data

INTRODUCTION

Electronic health records (EHRs) offer potential for population health surveillance but EHR-based surveillance measures require validation prior to use. We assessed the validity of obesity, smoking, depression, and influenza vaccination indicators from a new EHR surveillance system, the New York City (NYC) Macroscope. This report is the second in a 3-part series describing the development and validation of the NYC Macroscope. The first report describes in detail the infrastructure underlying the NYC Macroscope; design decisions that were made to maximize data quality; characteristics of the population sampled; completeness of data collected; and lessons learned from doing this work. This second report, which addresses concerns related to sampling bias and data quality, describes the methods used to evaluate the validity and robustness of NYC Macroscope prevalence estimates; presents validation results for estimates of obesity, smoking, depression and influenza vaccination; and discusses the implications of our findings for NYC and for other jurisdictions embarking on similar work. The third report applies the same validation methods described in this report to metabolic outcomes, including the prevalence, treatment and control of diabetes, hypertension and hyperlipidemia.

METHODS

NYC Macroscope prevalence estimates, overall and stratified by sex and age group, were compared to reference survey estimates for adult New Yorkers who reported visiting a doctor in the past year. Agreement was evaluated against 5 a priori criteria. Sensitivity and specificity were assessed by examining individual EHR records in a subsample of 48 survey participants.

RESULTS

Among adult New Yorkers in care, the NYC Macroscope prevalence estimate for smoking (15.2%) fell between estimates from NYC HANES (17.7 %) and CHS (14.9%) and met all 5 a priori criteria. The NYC Macroscope obesity prevalence estimate (27.8%) also fell between the NYC HANES (31.3%) and CHS (24.7%) estimates, but met only 3 a priori criteria. Sensitivity and specificity exceeded 0.90 for both the smoking and obesity indicators. The NYC Macroscope estimates of depression and influenza vaccination prevalence were more than 10 percentage points lower than the estimates from either reference survey. While specificity was > 0.90 for both of these indicators, sensitivity was < 0.70.

DISCUSSION

Through this work we have demonstrated that EHR data from a convenience sample of providers can produce acceptable estimates of smoking and obesity prevalence among adult New Yorkers in care; gained a better understanding of the challenges involved in estimating depression prevalence from EHRs; and identified areas for additional research regarding estimation of influenza vaccination prevalence. We have also shared lessons learned about how EHR indicators should be constructed and offer methodologic suggestions for validating them.

CONCLUSIONS

This work adds to a rapidly emerging body of literature about how to define, collect and interpret EHR-based surveillance measures and may help guide other jurisdictions.