Design of a national retail data monitor for public health surveillance

Avenues for Strengthening PCORnet's Capacity to Advance Patient-Centered Economic Outcomes in Patient-Centered Outcomes Research (PCOR)

PCORnet, the National Patient-Centered Clinical Research Network, provides the ability to conduct prospective and observational pragmatic research by leveraging standardized, curated electronic health records data together with patient and stakeholder engagement. PCORnet is funded by the Patient-Centered Outcomes Research Institute (PCORI) and is composed of 8 Clinical Research Networks that incorporate at total of 79 health system "sites." As the network developed, linkage to commercial health plans, federal insurance claims, disease registries, and other data resources demonstrated the value in extending the networks infrastructure to provide a more complete representation of patient's health and lived experiences. Initially, PCORnet studies avoided direct economic comparative effectiveness as a topic. However, PCORI's authorizing law was amended in 2019 to allow studies to incorporate patient-centered economic outcomes in primary research aims. With PCORI's expanded scope and PCORnet's phase 3 beginning in January 2022, there are opportunities to strengthen the network's ability to support economic patient-centered outcomes research. This commentary will discuss approaches that have been incorporated to date by the network and point to opportunities for the network to incorporate economic variables for analysis, informed by patient and stakeholder perspectives. Topics addressed include: (1) data linkage infrastructure; (2) commercial health plan partnerships; (3) Medicare and Medicaid linkage; (4) health system billing-based benchmarking; (5) area-level measures; (6) individual-level measures; (7) pharmacy benefits and retail pharmacy data; and (8) the importance of transparency and engagement while addressing the biases inherent in linking real-world data sources.

Global Research on Syndromic Surveillance from 1993 to 2017: Bibliometric Analysis and Visualization

Syndromic Surveillance aims at analyzing medical data to detect clusters of illness or forecast disease outbreaks. Although the research in this field is flourishing in terms of publications, an insight of the global research output has been overlooked. This paper aims at analyzing the global scientific output of the research from 1993 to 2017. To this end, the paper uses bibliometric analysis and visualization to achieve its goal. Particularly, a data processing framework was proposed based on citation datasets collected from Scopus and Clarivate Analytics’ Web of Science Core Collection (WoSCC). The bibliometric method and Citespace were used to analyze the institutions, countries, and research areas as well as the current hotspots and trends. The preprocessed dataset includes 14,680 citation records. The analysis uncovered USA, England, Canada, France and Australia as the top five most productive countries publishing about Syndromic Surveillance. On the other hand, at the Pinnacle of academic institutions are the US Centers for Disease Control and Prevention (CDC). The reference co-citation analysis uncovered the common research venues and further analysis of the keyword cooccurrence revealed the most trending topics. The findings of this research will help in enriching the field with a comprehensive view of the status and future trends of the research on Syndromic Surveillance.

Association of over-the-counter pharmaceutical sales with influenza-like-illnesses to patient volume in an urgent care setting

We studied the association between OTC pharmaceutical sales and volume of patients with influenza-like-illnesses (ILI) at an urgent care center over one year. OTC pharmaceutical sales explain 36% of the variance in the patient volume, and each standard deviation increase is associated with 4.7 more patient visits to the urgent care center (p<0.0001). Cross-correlation function analysis demonstrated that OTC pharmaceutical sales are significantly associated with patient volume during non-flu season (p<0.0001), but only the sales of cough and cold (p<0.0001) and thermometer (p<0.0001) categories were significant during flu season with a lag of two and one days, respectively. Our study is the first study to demonstrate and measure the relationship between OTC pharmaceutical sales and urgent care center patient volume, and presents strong evidence that OTC sales predict urgent care center patient volume year round.

A method for estimating from thermometer sales the incidence of diseases that are symptomatically similar to influenza

Early detection and accurate characterization of disease outbreaks are important tasks of public health. Infectious diseases that present symptomatically like influenza (SLI), including influenza itself, constitute an important class of diseases that are monitored by public-health epidemiologists. Monitoring emergency department (ED) visits for presentations of SLI could provide an early indication of the presence, extent, and dynamics of such disease in the population. We investigated the use of daily over-the-counter thermometer-sales data to estimate daily ED SLI counts in Allegheny County (AC), Pennsylvania. We found that a simple linear model fits the data well in predicting daily ED SLI counts from daily counts of thermometer sales in AC. These results raise the possibility that this model could be applied, perhaps with adaptation, in other regions of the country, where commonly thermometer sales data are available, but daily ED SLI counts are not.

High-level specification of a proposed information architecture for support of a bioterrorism early-warning system

Current information systems for use in detecting bioterrorist attacks lack a consistent, overarching information architecture. An overview of the use of biological agents as weapons during a bioterrorist attack is presented. Proposed are the design, development, and implementation of a medical informatics system to mine pertinent databases, retrieve relevant data, invoke appropriate biostatistical and epidemiological software packages, and automatically analyze these data. The top-level information architecture is presented. Systems requirements and functional specifications for this level are presented. Finally, future studies are identified.

Conceptualising and creating a global learning health system

In any country the health sector is important in terms of human wellbeing and large in terms of economics. The health sector might therefore be expected to be a finely tuned enterprise, utilising corporate knowledge in a constant process of critically reviewing and improving its activities and processes. However, this is seldom the case. Health systems and practice are highly variable and lag behind research discovery. This contrasts strongly with commercial bodies, and particularly service industries, where the concept of the learning organisation is strongly seen as the key to optimisation. A learning organisation accesses for analytic purposes operational data, which though captured and recorded for day-to-day transactions at the customer level, become also the basis of understanding changes in both demand and delivery process. In health care, the concept of the learning organisation is well grounded ethically. Anything which can improve health, including understanding of optimal care delivery processes and how to improve longer term outcomes, should be seized upon to drive service improvement - but currently this occurs haphazardly. The limitations of paper-based systems, priority given to digitalization of financial transactions, concerns about electronic data insecurity, and other factors have inhibited progress towards organisational learning at a national scale. But in recent years, new means of capturing, managing, and exchanging data have created new opportunities, while ever increasing pressures on health systems have produced strengthened incentive. In the United States, the current policy and investment impetus to electronic health records and concomitantly their 'meaningful use' create opportunities to build the foundations for data re-use for corporate learning - and thus for societal gain. In Europe and other settings there are islands of innovation, but not yet a coherent culture or impetus to build foundations for a learning health system. This paper considers how to move forward, in the light of the urgent need for smarter health systems where experience becomes the fuel for rapid improvement, and best practices are routinely identified and applied.

Collecting and sharing data for population health: a new paradigm

Health information technology (IT) has great potential to transform health care and inform population health goals in clinical research, quality measurement, and public safety. To fully realize the benefits of health IT for population health, we must focus on new models that maximize efficiency, encourage rapid learning, and protect patients' privacy. In this paper we explore the advantages of a networked model for analyzing population health information, providing several examples. Although broadening the use of networked models is challenging, the societal benefits of a networked model merit continued exploration and the development of workable solutions.

The EpiCanvas infectious disease weather map: an interactive visual exploration of temporal and spatial correlations

Advances in surveillance science have supported public health agencies in tracking and responding to disease outbreaks. Increasingly, epidemiologists have been tasked with interpreting multiple streams of heterogeneous data arising from varied surveillance systems. As a result public health personnel have experienced an overload of plots and charts as information visualization techniques have not kept pace with the rapid expansion in data availability. This study sought to advance the science of public health surveillance data visualization by conceptualizing a visual paradigm that provides an 'epidemiological canvas' for detection, monitoring, exploration and discovery of regional infectious disease activity and developing a software prototype of an 'infectious disease weather map'. Design objectives were elucidated and the conceptual model was developed using cognitive task analysis with public health epidemiologists. The software prototype was pilot tested using retrospective data from a large, regional pediatric hospital, and gastrointestinal and respiratory disease outbreaks were re-created as a proof of concept.

A method for selecting and monitoring medication sales for surveillance of gastroenteritis

PURPOSE

Monitoring appropriate categories of medication sales can provide early warning of certain disease outbreaks. This paper presents a methodology for choosing and monitoring medication sales relevant for the surveillance of gastroenteritis and assesses the operational characteristics of the selected medications for early warning.

METHODS

Acute diarrhoea incidences in mainland France were obtained from the Sentinelles network surveillance system for the period 2000-2009. Medication sales grouped by therapeutic classes were obtained on the same period. Hierarchical clustering was used to select therapeutic classes correlating with disease incidence over the period. Alert thresholds were defined for the selected therapeutic classes. Single and multiple voter algorithms were investigated for outbreak detection based on sales crossing the thresholds. Sensitivity and specificity were calculated respective to known outbreaks periods.

RESULTS

Four therapeutic classes were found to cluster with acute diarrhoea incidence. The therapeutic class other antiemetic and antinauseants had the best sensitivity (100%) and timeliness (1.625 weeks before official alerts), for a false alarm rate of 5%. Multiple voter algorithm was the most efficient with the rule: 'Emit an outbreak alert when at least three therapeutic classes are over their threshold' (sensitivity 100%, specificity 95%, timeliness 1.750 weeks before official alerts).

CONCLUSIONS

The presented method allowed selection of relevant therapeutic classes for surveillance of a specific condition. Multiple voter algorithm based on several therapeutic classes performed slightly better than the best therapeutic class alone, while improving robustness against abrupt changes occurring in a single therapeutic class.

Combining free text and structured electronic medical record entries to detect acute respiratory infections

BACKGROUND

The electronic medical record (EMR) contains a rich source of information that could be harnessed for epidemic surveillance. We asked if structured EMR data could be coupled with computerized processing of free-text clinical entries to enhance detection of acute respiratory infections (ARI).

METHODOLOGY

A manual review of EMR records related to 15,377 outpatient visits uncovered 280 reference cases of ARI. We used logistic regression with backward elimination to determine which among candidate structured EMR parameters (diagnostic codes, vital signs and orders for tests, imaging and medications) contributed to the detection of those reference cases. We also developed a computerized free-text search to identify clinical notes documenting at least two non-negated ARI symptoms. We then used heuristics to build case-detection algorithms that best combined the retained structured EMR parameters with the results of the text analysis.

PRINCIPAL FINDINGS

An adjusted grouping of diagnostic codes identified reference ARI patients with a sensitivity of 79%, a specificity of 96% and a positive predictive value (PPV) of 32%. Of the 21 additional structured clinical parameters considered, two contributed significantly to ARI detection: new prescriptions for cough remedies and elevations in body temperature to at least 38°C. Together with the diagnostic codes, these parameters increased detection sensitivity to 87%, but specificity and PPV declined to 95% and 25%, respectively. Adding text analysis increased sensitivity to 99%, but PPV dropped further to 14%. Algorithms that required satisfying both a query of structured EMR parameters as well as text analysis disclosed PPVs of 52-68% and retained sensitivities of 69-73%.

CONCLUSION

Structured EMR parameters and free-text analyses can be combined into algorithms that can detect ARI cases with new levels of sensitivity or precision. These results highlight potential paths by which repurposed EMR information could facilitate the discovery of epidemics before they cause mass casualties.

A service-oriented healthcare message alerting architecture in an Asia medical center: a case study

This paper illustrates how our development team has used some information technologies to let physicians obtain an instant abnormal laboratory result report for critical patient care services. We have implemented a healthcare message alerting system (HMAS) on a healthcare short message service (HSMS) engine and the distributed healthcare-oriented service environment (DiHOSE) in the National Taiwan University Hospital (NTUH). The HSMS engine has a general interface for all applications which could easily send any kind of alerting messages. Fundamentally, the DiHOSE uses HL7 standard formats to process the information exchange behaviors and can be flexibly extended for reasonable user requirements. The disease surveillance subsystem is an integral part of NTUH new hospital information system which is based on DiHOSE and the disease surveillance subsystem would send alerting messages through the HSMS engine. The latest cell phone message alerting subsystem, a case study, in NTUH proved that the DiHOSE could integrate the user required functions without much work. We concluded that both HSMS and DiHOSE can generalize and extend application demands efficiently.

Mining aggregates of over-the-counter products for syndromic surveillance

There has recently been a surge of research efforts aimed at very early detection of disease outbreaks. An important strategy for improving the timeliness of outbreak detection is to identify signals that occur early in the epidemic process. We have developed a novel algorithm to identify aggregates of "similar" over-the-counter products that have strong association with a given disease. This paper discusses the proposed algorithm and reports the results of an evaluation experiment. The experimental results show that this algorithm holds promise for discovering product aggregates with outbreak detection performance that is superior to that of predefined categories. We also found that the products extracted by the proposed algorithm were more strongly correlated with the disease data than the standard predefined product categories, while also being more strongly correlated with each other than the products in any predefined category.

Evaluation of preprocessing techniques for chief complaint classification

OBJECTIVE

To determine whether preprocessing chief complaints before automatically classifying them into syndromic categories improves classification performance.

METHODS

We preprocessed chief complaints using two preprocessors (CCP and EMT-P) and evaluated whether classification performance increased for a probabilistic classifier (CoCo) or for a keyword-based classifier (modification of the NYC Department of Health and Mental Hygiene chief complaint coder (KC)).

RESULTS

CCP exhibited high accuracy (85%) in preprocessing chief complaints but only slightly improved CoCo's classification performance for a few syndromes. EMT-P, which splits chief complaints into multiple problems, substantially increased CoCo's sensitivity for all syndromes. Preprocessing with CCP or EMT-P only improved KC's sensitivity for the Constitutional syndrome.

CONCLUSION

Evaluation of preprocessing systems should not be limited to accuracy of the preprocessor but should include the effect of preprocessing on syndromic classification. Splitting chief complaints into multiple problems before classification is important for CoCo, but other preprocessing steps only slightly improved classification performance for CoCo and a keyword-based classifier.

An evaluation of biosurveillance grid--dynamic algorithm distribution across multiple computer nodes

Performing fast data analysis to detect disease outbreaks plays a critical role in real-time biosurveillance. In this paper, we described and evaluated an Algorithm Distribution Manager Service (ADMS) based on grid technologies, which dynamically partition and distribute detection algorithms across multiple computers. We compared the execution time to perform the analysis on a single computer and on a grid network (3 computing nodes) with and without using dynamic algorithm distribution. We found that algorithms with long runtime completed approximately three times earlier in distributed environment than in a single computer while short runtime algorithms performed worse in distributed environment. A dynamic algorithm distribution approach also performed better than static algorithm distribution approach. This pilot study shows a great potential to reduce lengthy analysis time through dynamic algorithm partitioning and parallel processing, and provides the opportunity of distributing algorithms from a client to remote computers in a grid network.

Automated real time constant-specificity surveillance for disease outbreaks

Background

For real time surveillance, detection of abnormal disease patterns is based on a difference between patterns observed, and those predicted by models of historical data. The usefulness of outbreak detection strategies depends on their specificity; the false alarm rate affects the interpretation of alarms.

Results

We evaluate the specificity of five traditional models: autoregressive, Serfling, trimmed seasonal, wavelet-based, and generalized linear. We apply each to 12 years of emergency department visits for respiratory infection syndromes at a pediatric hospital, finding that the specificity of the five models was almost always a non-constant function of the day of the week, month, and year of the study (p < 0.05). We develop an outbreak detection method, called the expectation-variance model, based on generalized additive modeling to achieve a constant specificity by accounting for not only the expected number of visits, but also the variance of the number of visits. The expectation-variance model achieves constant specificity on all three time scales, as well as earlier detection and improved sensitivity compared to traditional methods in most circumstances.

Conclusion

Modeling the variance of visit patterns enables real-time detection with known, constant specificity at all times. With constant specificity, public health practitioners can better interpret the alarms and better evaluate the cost-effectiveness of surveillance systems.

Evaluating detection of an inhalational anthrax outbreak

Timely detection of an inhalational anthrax outbreak is critical for clinical and public health management. Syndromic surveillance has received considerable investment, but little is known about how it will perform relative to routine clinical case finding for detection of an inhalational anthrax outbreak. We conducted a simulation study to compare clinical case finding with syndromic surveillance for detection of an outbreak of inhalational anthrax. After simulated release of 1 kg of anthrax spores, the proportion of outbreaks detected first by syndromic surveillance was 0.59 at a specificity of 0.9 and 0.28 at a specificity of 0.975. The mean detection benefit of syndromic surveillance was 1.0 day at a specificity of 0.9 and 0.32 days at a specificity of 0.975. When syndromic surveillance was sufficiently sensitive to detect a substantial proportion of outbreaks before clinical case finding, it generated frequent false alarms.

Unsupervised clustering of over-the-counter healthcare products into product categories

A general problem in biosurveillance is finding appropriate aggregates of elemental data to monitor for the detection of disease outbreaks. We developed an unsupervised clustering algorithm for aggregating over-the-counter healthcare (OTC) products into categories. This algorithm employs MCMC over hundreds of parameters in a Bayesian model to place products into clusters. Despite the high dimensionality, it still performs fast on hundreds of time series. The procedure was able to uncover a clinically significant distinction between OTC products intended for the treatment of allergy and OTC products intended for the treatment of cough, cold, and influenza symptoms.

Challenges faced by hospital healthcare workers in using a syndrome-based surveillance system during the 2003 outbreak of severe acute respiratory syndrome in Taiwan

Because the severe acute respiratory syndrome (SARS) outbreak in Taiwan in 2003 was worsened by hospital infections, we analyzed 229 questionnaires (84.8% of 270 sent) completed by surveyed healthcare workers who cared for patients with SARS in 3 types of hospitals, to identify surveillance problems. Atypical clinical presentation was the most often reported problem, regardless of hospital type, which strongly indicates that more timely syndromic surveillance was needed.

Surveillance for Newly Emerging Viruses

Surveillance is a fundamental tool for public health, producing information to guide actions. Modern surveillance tends to follow health measures such as the incidence of a disease or syndrome or even the occurrence of health-related behaviors. There are many reasons for conducting surveillance, and the data collected and the approach taken to analyzing those data are both influenced by the overall goal of a surveillance system. Surveillance systems aims mainly at detection also provide information that may be useful for other purposes. The goal of detecting an outbreak of a newly emerging virus, places specific demands on the type of data collected and the types of analysis performed. All approaches to surveillance share some common principles. While some of the underlying methods used in public health surveillance have evolved considerably in recent years, the general approach to surveillance has remained relatively constant. At a fundamental level, surveillance aims to (1) identify individual cases, (2) detect population patterns in identified cases, and then (3) convey information to decision-makers about population health patterns.

Lumbar puncture ordering and results in the pediatric population: a promising data source for surveillance systems

BACKGROUND

The Centers for Disease Control and Prevention is incorporating laboratory data into real-time surveillance systems. When normal patterns of laboratory test orders and results are modeled, aberrations can be detected. Because many test orders are available electronically well before results, atypical patterns of test ordering may signal outbreaks.

OBJECTIVES

The authors sought to characterize baseline patterns in the ordering and early results of lumbar punctures, motivated by the possibility of using these data for real-time surveillance for early detection of meningitis or encephalitis outbreaks.

METHODS

Retrospective cohorts of pediatric emergency department patients at a single hospital (1993-2003) and from the National Hospital and Ambulatory Medical Care Survey (1992-2000) were used for analysis.

RESULTS

Test ordering exhibits seasonal patterns, with monthly peaks in January and August (p < 0.0001). For the hospital cohort, the rate of cerebrospinal fluid pleocytosis exhibits seasonal patterns (p < 0.0001), with a peak from August to October. This is strongly associated with the rate and pattern of clinical neurologic disease (p < 0.0001). A long-term secular decline in daily test ordering is evident, dropping from 5.3 to 2.9 in the hospital sample, and from 371.8 to 185.3 in the national sample (p < 0.001). The long-term rate of pleocytosis has declined (p < 0.0001), though the yield of testing for pleocytosis has improved (p = 0.0104).

CONCLUSIONS

Laboratory test patterns correspond with those of clinical disease and are a promising source of surveillance data. Using such data for real-time monitoring requires specific adjustments for patient age, periodicities, and secular trends.

Diarrheal illness detected through syndromic surveillance after a massive power outage: New York City, August 2003

OBJECTIVES

We investigated increases in diarrheal illness detected through syndromic surveillance after a power outage in New York City on August 14, 2003.

METHODS

The New York City Department of Health and Mental Hygiene uses emergency department, pharmacy, and absentee data to conduct syndromic surveillance for diarrhea. We conducted a case-control investigation among patients presenting during August 16 to 18, 2003, to emergency departments that participated in syndromic surveillance. We compared risk factors for diarrheal illness ascertained through structured telephone interviews for case patients presenting with diarrheal symptoms and control patients selected from a stratified random sample of nondiarrheal patients.

RESULTS

Increases in diarrhea were detected in all data streams. Of 758 patients selected for the investigation, 301 (40%) received the full interview. Among patients 13 years and older, consumption of meat (odds ratio [OR]=2.7, 95% confidence interval [CI]=1.2, 6.1) and seafood (OR=4.8; 95% CI=1.6, 14) between the power outage and symptom onset was associated with diarrheal illness.

CONCLUSIONS

Diarrhea may have resulted from consumption of meat or seafood that spoiled after the power outage. Syndromic surveillance enabled prompt detection and systematic investigation of citywide illness that would otherwise have gone undetected.

Using Ontario's "Telehealth" health telephone helpline as an early-warning system: a study protocol

Background

The science of syndromic surveillance is still very much in its infancy. While a number of syndromic surveillance systems are being evaluated in the US, very few have had success thus far in predicting an infectious disease event. Furthermore, to date, the majority of syndromic surveillance systems have been based primarily in emergency department settings, with varying levels of enhancement from other data sources. While research has been done on the value of telephone helplines on health care use and patient satisfaction, very few projects have looked at using a telephone helpline as a source of data for syndromic surveillance, and none have been attempted in Canada. The notable exception to this statement has been in the UK where research using the national NHS Direct system as a syndromic surveillance tool has been conducted.

Methods/design

The purpose of our proposed study is to evaluate the effectiveness of Ontario's telephone nursing helpline system as a real-time syndromic surveillance system, and how its implementation, if successful, would have an impact on outbreak event detection in Ontario. Using data collected retrospectively, all "reasons for call" and assigned algorithms will be linked to a syndrome category. Using different analytic methods, normal thresholds for the different syndromes will be ascertained. This will allow for the evaluation of the system's sensitivity, specificity and positive predictive value. The next step will include the prospective monitoring of syndromic activity, both temporally and spatially.

Discussion

As this is a study protocol, there are currently no results to report. However, this study has been granted ethical approval, and is now being implemented. It is our hope that this syndromic surveillance system will display high sensitivity and specificity in detecting true outbreaks within Ontario, before they are detected by conventional surveillance systems. Future results will be published in peer-reviewed journals so as to contribute to the growing body of evidence on syndromic surveillance, while also providing an non US-centric perspective.

Validation of syndromic surveillance for respiratory infections

Study objective

A key public health question is whether syndromic surveillance data provide early warning of infectious outbreaks. One cause for skepticism is that biological correlates of the administrative and clinical data used in these systems have not been rigorously assessed. This study measures the value of respiratory data currently used in syndromic surveillance systems to detect respiratory infections by comparing it against criterion standard viral testing within a pediatric population.

Methods

We conducted a longitudinal study with prospective validation in the emergency department (ED) of a tertiary care children’s hospital. Children aged 7 years or younger who presented with a respiratory syndrome or who were tested for respiratory syncytial virus (RSV), influenza virus, parainfluenza virus, adenovirus, or enterovirus between January 1993 and June 2004 were included. We assessed the predictive ability of the viral tests by fitting generalized linear models to respiratory syndrome counts.

Results

Of 582,635 patient visits, 89,432 (15.4%) were for respiratory syndromes, and of these, 7,206 (8.1%) patients were tested for the viruses of interest. RSV was significantly related to respiratory syndrome counts (adjusted rate ratio [RR] 1.33; 95% confidence interval [CI] 1.04 to 1.71). In multivariate models including all viruses tested, influenza virus was also a significant predictor of respiratory syndrome counts (RR 1.47; 95% CI 1.03 to 2.10). This model accounted for 81.6% of the observed variability in respiratory syndrome counts.

Conclusion

Respiratory syndromic surveillance data strongly correlate with virologic test results in a pediatric population, providing evidence of the biologic validity of such surveillance systems. Real-time outbreak detection systems relying on syndromic data may be an important adjunct to the current set of public health systems for the detection and surveillance of respiratory infections.

Identifying pediatric age groups for influenza vaccination using a real-time regional surveillance system

Evidence is accumulating that universal vaccination of schoolchildren would reduce the transmission of influenza. The authors sought to identify target age groups within the pediatric population that develop influenza the earliest and are most strongly linked with mortality in the population. Patient visits for respiratory illness were monitored, using real-time syndromic surveillance systems, in six Massachusetts health-care settings, including ambulatory care sites and emergency departments at tertiary-care and community hospitals. Visits from January 1, 2000, to September 30, 2004, were segmented into age group subpopulations. Timeliness and prediction of each subpopulation were measured against pneumonia and influenza mortality in New England with time-series analyses and regression models. Study results show that patient age significantly influences timeliness (p = 0.026), with pediatric age groups arriving first (p < 0.001); children aged 3-4 years are consistently the earliest (p = 0.0058). Age also influences the degree of prediction of mortality (p = 0.036), with illness among children under age 5 years, compared with all other patients, most strongly associated with mortality (p < 0.001). Study findings add to a growing body of support for a strategy to vaccinate children older than the currently targeted age of 6-23 months and specifically suggest that there may be value in vaccinating preschool-age children.

High-fidelity injection detectability experiments: a tool for evaluating syndromic surveillance systems

INTRODUCTION

When public health surveillance systems are evaluated, CDC recommends that the expected sensitivity, specificity, and timeliness of surveillance systems be characterized for outbreaks of different sizes, etiologies, and geographic or demographic scopes. High-Fidelity Injection Detectability Experiments (HiFIDE) is a tool that health departments can use to compute these metrics for detection algorithms and surveillance data that they are using in their surveillance system.

OBJECTIVE

The objective of this study is to develop a tool that allows health departments to estimate the expected sensitivity, specificity, and timeliness of outbreak detection.

METHODS

HiFIDE extends existing semisynthetic injection methods by replacing geometrically shaped injects with injects derived from surveillance data collected during real outbreaks. These injects maintain the known relation between outbreak size and effect on surveillance data, which allows inferences to be made regarding the smallest outbreak that can be expected to be detectable.

RESULTS

An example illustrates the use of HiFIDE to analyze detectability of a waterborne Cryptosporidium outbreak in Washington, DC.

CONCLUSION

HiFIDE enables public health departments to perform system validations recommended by CDC. HiFIDE can be obtained for no charge for noncommercial use (http://www.hifide.org).

Design and Operation of State and Local Infectious Disease Surveillance Systems

Since 2001, increased attention has been focused on improving acute infectious disease surveillance systems. This article describes options for their design and operation. Systems designed primarily to detect individual cases of reportable diseases may differ from those designed to detect outbreaks or support design or evaluation of control programs. Timeliness, sensitivity, and predictive value of surveillance systems cannot all be maximized at the same time. Core activities of surveillance systems include collection, analysis, and dissemination of information about health events under surveillance. Doing these well requires attention to the mechanics of surveillance, such as making the health department accessible at all times to receive reports and provide consultation, and maintaining current directories of persons for dissemination of surveillance data, alerts, and recommendations. Rapid access to electronic representations of health events (eg, laboratory reports, patient records, or health care claims) provides great opportunities for more timely and complete surveillance. Some important information (eg, exposures, contacts) will still need to be collected directly from affected persons. One productive strategy is to collect core demographic and onset data on all cases and detailed clinical, exposure, and outcome data on a subset.

Key design elements of a data utility for national biosurveillance: event-driven architecture, caching, and Web service model

The National Retail Data Monitor (NRDM) has monitored over-the-counter (OTC) medication sales in the United States since December 2002. The NRDM collects data from over 18,600 retail stores and processes over 0.6 million sales records per day. This paper describes key architectural features that we have found necessary for a data utility component in a national biosurveillance system. These elements include event-driven architecture to provide analyses of data in near real time, multiple levels of caching to improve query response time, high availability through the use of clustered servers, scalable data storage through the use of storage area networks and a web-service function for interoperation with affiliated systems. The methods and architectural principles are relevant to the design of any production data utility for public health surveillance-systems that collect data from multiple sources in near real time for use by analytic programs and user interfaces that have substantial requirements for time-series data aggregated in multiple dimensions.

A multivariate procedure for identifying correlations between diagnoses and over-the-counter products from historical datasets

A general problem in biosurveillance is finding the optimal aggregates of more basic data to monitor for the detection of disease outbreaks. We developed a multivariate procedure for identifying the set of over-the-counter (OTC) healthcare products that correlates best with a set of diagnoses. To ensure that the procedure produces results that agree with clinical knowledge of diseases and (OTC) products, we applied it to a set of products and set of diagnoses for which the correlation was known to be high. Our hypothesis was that the model could achieve parsimony in the set of diagnoses that correlate with sales of pediatric electrolytes while still producing a high correlation. The procedure narrowed the set of diagnoses that correlate with pediatric electrolytes from 51 diagnoses to eight diagnoses. The correlation of the set of 51 diagnoses with electrolyte sales was 0.95 and the correlation of the set of 8 diagnoses with electrolytes was 0.96. We conclude that the procedure functions as intended and is suitable for further testing with other problems in finding optimal aggregates of OTC products, and more generally of other types of biosurveillance data, to monitor for the detection of various disease outbreaks.

An evaluation of three policies for updating product categories in the National Retail Data Monitor

A problem in biosurveillance is how frequently to update controlled vocabularies that identify various data elements such as laboratory tests and over-the-counter healthcare products. More frequent updates improve completeness of data captured over time, but introduction of new codes into a surveillance system may cause false alarms when codes are aggregated into analytic categories. We studied the effect of three policies for updating UPCs, the controlled vocabulary for over-the-counter healthcare products used by the National Retail Data Monitor. To compare different policies for updating, we analyzed historical data from two cities for the 18 product categories of the National Retail Data Monitor under annual, quarterly, or monthly UPC update policies. We measured the effect on data completeness and false alarm rate. We found that the monthly update policy had the highest data completeness and led to the fewest number of additional false alarms. Overall, monthly updating of UPCs was the superior policy.

Investigating public health emergency response information system initiatives in China

Infectious diseases pose a great danger to public health internationally. The outbreak of SARS has exposed China’s fragile public health system and its limited ability to detect and respond to emergencies in a timely and effective manner. In order to strengthen its capability of responding to future public health emergencies, China is developing a public health emergency response information system (PHERIS) to facilitate disease surveillance, detection, reporting, and response. The purpose of this study is to investigate the ongoing development of China’s PHERIS. This paper analyzes the problems of China’s existing public health system and describes the design and functionalities of PHERIS from both technical and managerial aspects.

How outbreaks of infectious disease are detected: a review of surveillance systems and outbreaks

To learn how outbreaks of infectious disease are detected and to describe the entities and information systems that together function to identify outbreaks in the U.S., the authors drew on multiple sources of information to create a description of existing surveillance systems and how they interact to detect outbreaks. The results of this analysis were summarized in a system diagram. The authors reviewed a sample of recent outbreaks to determine how they were detected, with reference to the system diagram. The de facto U.S. system for detection of outbreaks consists of five components: the clinical health care system, local/state health agencies, federal agencies, academic/professional organizations, and collaborating governmental organizations. Primary data collection occurs at the level of clinical health care systems and local health agencies. The review of a convenience sample of outbreaks showed that all five components of the system participated in aggregating, analyzing, and sharing data. The authors conclude that the current U.S. approach to detection of disease outbreaks is complex and involves many organizations interacting in a loosely coupled manner. State and local health departments and the health care system are major components in the detection of outbreaks.

Is NHS dentistry in crisis? 'Traffic light' maps of dentists distribution in England and Wales

BACKGROUND: 'Traffic light' (red-yellow-green) maps are potentially powerful tools for 'at a glance' problem detection, for optimising resource allocation/reallocation, setting priorities, and targeting interventions to areas most in need. The maps can be also used for administrative area comparisons and performance monitoring over time. Interactive Web versions of the maps can be generated with many handy features to further empower organisations and decision makers. Methodological issues to consider when creating 'traffic light' maps include hue thresholding, data timeliness and stability of administrative boundaries. RESULTS: We used 'traffic light' maps to study the distribution of dentists per 1,000 population in all 304 English Primary Care Trusts (PCTs) and 22 Welsh Local Health Boards (LHBs) using datasets of dentist numbers per PCT (as at 31 December 2002) and LHB (as at 26 February 2004) from the Dental Practice Board, and 2001 Census population figures for PCTs and LHBs from the Office for National Statistics. The overall NHS dentists per 1,000 population figures for England (0.374) and Wales (0.359) are low compared to many other countries, with less than 0.3 dentist per 1,000 people available to 24.1% of the total population of England (81 PCTs or 26.6% of all PCTs) and 26.1% of the total population of Wales (6 LHBs or 27.3% of all LHBs). A general shortage of NHS dentists can be observed at a glance across England and Wales on all the 'traffic light' maps in our study, even on those using a more "tolerant" classification and an additional orange-yellow class. The distribution of NHS dentists in England and Wales was also found to be not uniform, with some PCTs/LHBs, especially those located in some of the deprived or less populated urban and rural communities, suffering significantly more shortage of dentists than others (see http://healthcybermap.org/PCT/dentists/). These results confirm recent media reports of a shortage of NHS dentists in various parts of England and Wales. CONCLUSION: Suitable programmes are urgently needed to increase the numbers of NHS dentists across England and Wales. We have included a set of recommendations to dental health policymakers and planners, in addition to ideas for further work.

Implementing syndromic surveillance: a practical guide informed by the early experience

Syndromic surveillance refers to methods relying on detection of individual and population health indicators that are discernible before confirmed diagnoses are made. In particular, prior to the laboratory confirmation of an infectious disease, ill persons may exhibit behavioral patterns, symptoms, signs, or laboratory findings that can be tracked through a variety of data sources. Syndromic surveillance systems are being developed locally, regionally, and nationally. The efforts have been largely directed at facilitating the early detection of a covert bioterrorist attack, but the technology may also be useful for general public health, clinical medicine, quality improvement, patient safety, and research. This report, authored by developers and methodologists involved in the design and deployment of the first wave of syndromic surveillance systems, is intended to serve as a guide for informaticians, public health managers, and practitioners who are currently planning deployment of such systems in their regions.

Towards evidence-based, GIS-driven national spatial health information infrastructure and surveillance services in the United Kingdom

The term "Geographic Information Systems" (GIS) has been added to MeSH in 2003, a step reflecting the importance and growing use of GIS in health and healthcare research and practices. GIS have much more to offer than the obvious digital cartography (map) functions. From a community health perspective, GIS could potentially act as powerful evidence-based practice tools for early problem detection and solving. When properly used, GIS can: inform and educate (professionals and the public); empower decision-making at all levels; help in planning and tweaking clinically and cost-effective actions, in predicting outcomes before making any financial commitments and ascribing priorities in a climate of finite resources; change practices; and continually monitor and analyse changes, as well as sentinel events. Yet despite all these potentials for GIS, they remain under-utilised in the UK National Health Service (NHS). This paper has the following objectives: (1) to illustrate with practical, real-world scenarios and examples from the literature the different GIS methods and uses to improve community health and healthcare practices, e.g., for improving hospital bed availability, in community health and bioterrorism surveillance services, and in the latest SARS outbreak; (2) to discuss challenges and problems currently hindering the wide-scale adoption of GIS across the NHS; and (3) to identify the most important requirements and ingredients for addressing these challenges, and realising GIS potential within the NHS, guided by related initiatives worldwide. The ultimate goal is to illuminate the road towards implementing a comprehensive national, multi-agency spatio-temporal health information infrastructure functioning proactively in real time. The concepts and principles presented in this paper can be also applied in other countries, and on regional (e.g., European Union) and global levels.

Towards evidence-based, GIS-driven national spatial health information infrastructure and surveillance services in the United Kingdom

The term "Geographic Information Systems" (GIS) has been added to MeSH in 2003, a step reflecting the importance and growing use of GIS in health and healthcare research and practices. GIS have much more to offer than the obvious digital cartography (map) functions. From a community health perspective, GIS could potentially act as powerful evidence-based practice tools for early problem detection and solving. When properly used, GIS can: inform and educate (professionals and the public); empower decision-making at all levels; help in planning and tweaking clinically and cost-effective actions, in predicting outcomes before making any financial commitments and ascribing priorities in a climate of finite resources; change practices; and continually monitor and analyse changes, as well as sentinel events. Yet despite all these potentials for GIS, they remain under-utilised in the UK National Health Service (NHS). This paper has the following objectives: (1) to illustrate with practical, real-world scenarios and examples from the literature the different GIS methods and uses to improve community health and healthcare practices, e.g., for improving hospital bed availability, in community health and bioterrorism surveillance services, and in the latest SARS outbreak; (2) to discuss challenges and problems currently hindering the wide-scale adoption of GIS across the NHS; and (3) to identify the most important requirements and ingredients for addressing these challenges, and realising GIS potential within the NHS, guided by related initiatives worldwide. The ultimate goal is to illuminate the road towards implementing a comprehensive national, multi-agency spatio-temporal health information infrastructure functioning proactively in real time. The concepts and principles presented in this paper can be also applied in other countries, and on regional (e.g., European Union) and global levels.

Technical description of RODS: a real-time public health surveillance system

This report describes the design and implementation of the Real-time Outbreak and Disease Surveillance (RODS) system, a computer-based public health surveillance system for early detection of disease outbreaks. Hospitals send RODS data from clinical encounters over virtual private networks and leased lines using the Health Level 7 (HL7) message protocol. The data are sent in real time. RODS automatically classifies the registration chief complaint from the visit into one of seven syndrome categories using Bayesian classifiers. It stores the data in a relational database, aggregates the data for analysis using data warehousing techniques, applies univariate and multivariate statistical detection algorithms to the data, and alerts users of when the algorithms identify anomalous patterns in the syndrome counts. RODS also has a Web-based user interface that supports temporal and spatial analyses. RODS processes sales of over-the-counter health care products in a similar manner but receives such data in batch mode on a daily basis. RODS was used during the 2002 Winter Olympics and currently operates in two states-Pennsylvania and Utah. It has been and continues to be a resource for implementing, evaluating, and applying new methods of public health surveillance.

Detection of pediatric respiratory and diarrheal outbreaks from sales of over-the-counter electrolyte products

OBJECTIVE

To determine whether sales of electrolyte products contain a signal of outbreaks of respiratory and diarrheal disease in children and, if so, how much earlier a signal relative to hospital diagnoses.

DESIGN

Retrospective analysis was conducted of sales of electrolyte products and hospital diagnoses for six urban regions in three states for the period 1998 through 2001.

MEASUREMENTS

Presence of signal was ascertained by measuring correlation between electrolyte sales and hospital diagnoses and the temporal relationship that maximized correlation. Earliness was the difference between the date that the exponentially weighted moving average (EWMA) method first detected an outbreak from sales and the date it first detected the outbreak from diagnoses. The coefficient of determination (r2) measured how much variance in earliness resulted from differences in sales' and diagnoses' signal strengths.

RESULTS

The correlation between electrolyte sales and hospital diagnoses was 0.90 (95% CI, 0.87-0.93) at a time offset of 1.7 weeks (95% CI, 0.50-2.9), meaning that sales preceded diagnoses by 1.7 weeks. EWMA with a nine-sigma threshold detected the 18 outbreaks on average 2.4 weeks (95% CI, 0.1-4.8 weeks) earlier from sales than from diagnoses. Twelve outbreaks were first detected from sales, four were first detected from diagnoses, and two were detected simultaneously. Only 26% of variance in earliness was explained by the relative strength of the sales and diagnoses signals (r2 = 0.26).

CONCLUSION

Sales of electrolyte products contain a signal of outbreaks of respiratory and diarrheal diseases in children and usually are an earlier signal than hospital diagnoses.