Assessing and Improving Data Integrity in Web-Based Surveys: Comparison of Fraud Detection Systems in a COVID-19 Study

BACKGROUND

Web-based surveys increase access to study participation and improve opportunities to reach diverse populations. However, web-based surveys are vulnerable to data quality threats, including fraudulent entries from automated bots and duplicative submissions. Widely used proprietary tools to identify fraud offer little transparency about the methods used, effectiveness, or representativeness of resulting data sets. Robust, reproducible, and context-specific methods of accurately detecting fraudulent responses are needed to ensure integrity and maximize the value of web-based survey research.

OBJECTIVE

This study aims to describe a multilayered fraud detection system implemented in a large web-based survey about COVID-19 attitudes, beliefs, and behaviors; examine the agreement between this fraud detection system and a proprietary fraud detection system; and compare the resulting study samples from each of the 2 fraud detection methods.

METHODS

The PhillyCEAL Common Survey is a cross-sectional web-based survey that remotely enrolled residents ages 13 years and older to assess how the COVID-19 pandemic impacted individuals, neighborhoods, and communities in Philadelphia, Pennsylvania. Two fraud detection methods are described and compared: (1) a multilayer fraud detection strategy developed by the research team that combined automated validation of response data and real-time verification of study entries by study personnel and (2) the proprietary fraud detection system used by the Qualtrics (Qualtrics) survey platform. Descriptive statistics were computed for the full sample and for responses classified as valid by 2 different fraud detection methods, and classification tables were created to assess agreement between the methods. The impact of fraud detection methods on the distribution of vaccine confidence by racial or ethnic group was assessed.

RESULTS

Of 7950 completed surveys, our multilayer fraud detection system identified 3228 (40.60%) cases as valid, while the Qualtrics fraud detection system identified 4389 (55.21%) cases as valid. The 2 methods showed only "fair" or "minimal" agreement in their classifications (κ=0.25; 95% CI 0.23-0.27). The choice of fraud detection method impacted the distribution of vaccine confidence by racial or ethnic group.

CONCLUSIONS

The selection of a fraud detection method can affect the study's sample composition. The findings of this study, while not conclusive, suggest that a multilayered approach to fraud detection that includes conservative use of automated fraud detection and integration of human review of entries tailored to the study's specific context and its participants may be warranted for future survey research.

Validation of the Canadian Institute for Health Information Diagnostic Codes for Benign Gynaecologic Surgery

We investigated the validity of the 10th Revision Canadian modification of International Statistical Classification of Disease and Related Health Problems (ICD-10-CA) diagnostic codes for surgery for benign gynaecologic conditions in the Canadian Institute for Health Information Discharge Abstract Database (CIHI-DAD), the main source of routinely collected data in Canada. Reabstracted data from patient charts was compared to ICD-10-CA codes and measures of validity were calculated with 95% confidence intervals. A total of 1068 procedures were identified. More objective, structural diagnoses (fibroids, prolapse) had higher sensitivity and near-perfect Kappa coefficients, while more subjective, symptomatic diagnoses (abnormal uterine bleeding, pelvic pain) had lower sensitivity and moderate-substantial Kappa coefficients. Specificity, positive predictive values, and negative predictive values were generally high for all diagnoses. These findings support the use of CIHI-DAD data for gynaecologic research.

Ontology-Based AI Design Patterns and Constraints in Cancer Registry Data Validation

Data validation in cancer registration is a critical operation but is resource-intensive and has traditionally depended on proprietary software. Ontology-based AI is a novel approach utilising machine reasoning based on axioms formally described in description logic. This is a different approach from deep learning AI techniques but not exclusive of them. The advantage of the ontology approach lies in its ability to address a number of challenges concurrently. The disadvantages relate to computational costs, which increase with language expressivity and the size of data sets, and class containment restrictions imposed by description logics. Both these aspects would benefit from the availability of design patterns, which is the motivation behind this study. We modelled the European cancer registry data validation rules in description logic using a number of design patterns and showed the viability of the approach. Reasoning speeds are a limiting factor for large cancer registry data sets comprising many hundreds of thousands of records, but these can be offset to a certain extent by developing the ontology in a modular way. Data validation is also a highly parallelisable process. Important potential future work in this domain would be to identify and optimise reusable design patterns, paying particular attention to avoiding any unintended reasoning efficiency hotspots.

An ontology design for validating childhood cancer registry data

Ontologies can provide a valuable role in the work of cancer registration, particularly as a tool for managing and navigating the various classification systems and coding rules. Further advantages accrue from the ability to formalise the coding rule base using description logics and thereby benefit from the associated automatic reasoning functionality. Drawing from earlier work that showed the viability of applying ontologies in the data validation tasks of cancer registries, an ontology was created using a modular approach to handle the specific checks for childhood cancers. The ontology was able to handle successfully the various inter-variable checks using the axiomatic constructs of the web ontology language. Application of an ontological approach for data validation can greatly simplify the maintenance of the coding rules and facilitate the federation of any centralised validation process to the local level. It also provides an improved means of visualising the rule interdependencies from different perspectives. Performance of the automatic reasoning process can be a limiting issue for very large datasets and will be a focus for future work. Results are provided showing how the ontology is able to validate cancer case records typical for childhood tumours.

Blockchain-Based Data Access Control and Key Agreement System in IoT Environment

Recently, with the increasing application of the Internet of Things (IoT), various IoT environments such as smart factories, smart homes, and smart grids are being generated. In the IoT environment, a lot of data are generated in real time, and the generated IoT data can be used as source data for various services such as artificial intelligence, remote medical care, and finance, and can also be used for purposes such as electricity bill generation. Therefore, data access control is required to grant access rights to various data users in the IoT environment who need such IoT data. In addition, IoT data contain sensitive information such as personal information, so privacy protection is also essential. Ciphertext-policy attribute-based encryption (CP-ABE) technology has been utilized to address these requirements. Furthermore, system structures applying blockchains with CP-ABE are being studied to prevent bottlenecks and single failures of cloud servers, as well as to support data auditing. However, these systems do not stipulate authentication and key agreement to ensure the security of the data transmission process and data outsourcing. Accordingly, we propose a data access control and key agreement scheme using CP-ABE to ensure data security in a blockchain-based system. In addition, we propose a system that can provide data nonrepudiation, data accountability, and data verification functions by utilizing blockchains. Both formal and informal security verifications are performed to demonstrate the security of the proposed system. We also compare the security, functional aspects, and computational and communication costs of previous systems. Furthermore, we perform cryptographic calculations to analyze the system in practical terms. As a result, our proposed protocol is safer against attacks such as guessing attacks and tracing attacks than other protocols, and can provide mutual authentication and key agreement functions. In addition, the proposed protocol is more efficient than other protocols, so it can be applied to practical IoT environments.

Evaluating the Accuracy of the Annual Statewide Kindergarten Vaccination Survey Submitted to Public Health Authorities by School Systems, Oklahoma, 2018

Recent trends in vaccine hesitancy have brought to light the importance of using accurate school vaccination data. This study evaluated the accuracy of a pilot statewide kindergarten vaccination survey in Oklahoma. School vaccination and exemption data were collected from November 2017 to April 2018 via the Research Electronic Data Capture system. A multivariable linear regression model was used to evaluate the relationship between students who are up to date for all vaccines comparing school reported and Oklahoma State Department of Health-validated data. Adjusted vaccination data were overestimated by 1.0% among public schools and 3.3% among private schools. These results were validated by a random audit of participating schools finding the school-reported vaccination data to be overestimated by 0.6% compared to true student immunization records on file. Our analysis indicates that school-reported vaccination data are sufficiently valid. Immunization record audits provide confidence in available data, which drives evidence-based decision-making.

Implementation of resource use measures in Medicare Advantage

OBJECTIVE

To complement the previously illustrated method to measure resource use in Medicare Advantage (MA) using Encounter data and provide technical details and SAS code to validate Encounter data and implement resource use measures in MA.

DATA SOURCES

2015-2018 MA Encounter, Medicare Provider Analysis and Review (MedPAR), Healthcare Effectiveness Data and Information System (HEDIS), and Traditional Medicare (TM) claims data.

STUDY DESIGN

Secondary data analysis.

DATA COLLECTION/EXTRACTION METHODS

We select MA contracts with high data completeness (≤10% missing hospital stays in Encounter data and ≤±10% difference in ambulatory and emergency department visits between Encounter and HEDIS data). We randomly sample TM beneficiaries with a similar geographic distribution as MA enrollees in the selected contracts. We develop standardized prices of services using TM payments, and we measure MA resource use for inpatient, outpatient, Part D, and hospice services.

PRINCIPAL FINDINGS

We report identifiers/names of contracts with high data completeness. We provide SAS code to manage Encounter data, develop standardized prices, and measure MA resource use.

CONCLUSIONS

Greater use and validation of Encounter data can help improve data quality. Our results can be used to inform studies using Encounter data to learn about MA performance.

Evaluating the accuracy and biological meaning of visits to RFID-enabled bird feeders using video

Radio-frequency identification (RFID) technology has gained popularity in ornithological studies as a way to collect large quantities of data to answer specific biological questions, but few published studies report methodologies used for validating the accuracy of RFID data. Further, connections between the RFID data and the behaviors of interest in a study are not always clearly established. These methodological deficiencies may seriously impact a study's results and subsequent interpretation. We built RFID-equipped bird feeders and mounted them at three sites in Tompkins County, New York. We deployed passive integrated transponder tags on black-capped chickadees, tufted titmice, and white-breasted nuthatches and used a GoPro video camera to record the three tagged species at the feeders. We then reviewed the video to determine the accuracy of the RFID reader and understand the birds' behavior at the feeders. We found that our RFID system recorded only 34.2% of all visits by tagged birds (n = 237) and that RFID detection increased with the length of a visit. We also found that our three tagged species and two other species that visited the feeders, American goldfinch and hairy woodpecker, retrieved food in 79.5% of their visits. Chickadees, titmice, nuthatches, and woodpeckers spent, on average, 2.3 s at feeders to collect one seed per visit. In contrast, goldfinches spent an average of 9.0 s at feeders and consumed up to 30 seeds per visit. Our results demonstrate the importance of confirming detection accuracy and that video can be used to identify behavioral characteristics associated with an RFID reader's detections. This simple-yet time-intensive-method for assessing the accuracy and biological meaning of RFID data is useful for ornithological studies but can be used in research focusing on various taxa and study systems.

Data Ecosystems for Scientific Experiments: Managing Combustion Experiments and Simulation Analyses in Chemical Engineering

The development of scientific predictive models has been of great interest over the decades. A scientific model is capable of forecasting domain outcomes without the necessity of performing expensive experiments. In particular, in combustion kinetics, the model can help improving the combustion facilities and the fuel efficiency reducing the pollutants. At the same time, the amount of available scientific data has increased and helped speeding up the continuous cycle of model improvement and validation. This has also opened new opportunities for leveraging a large amount of data to support knowledge extraction. However, experiments are affected by several data quality problems since they are a collection of information over several decades of research, each characterized by different representation formats and reasons of uncertainty. In this context, it is necessary to develop an automatic data ecosystem capable of integrating heterogeneous information sources while maintaining a quality repository. We present an innovative approach to data quality management from the chemical engineering domain, based on an available prototype of a scientific framework, SciExpeM, which has been significantly extended. We identified a new methodology from the model development research process that systematically extracts knowledge from the experimental data and the predictive model. In the paper, we show how our general framework could support the model development process, and save precious research time also in other experimental domains with similar characteristics, i.e., managing numerical data from experiments.

Quality and Completeness of Myocardial Infarction Recording in Clinical Practice Research Datalink Aurum

Background

Validation studies of the Clinical Practice Research Datalink (CPRD) Aurum database in the UK are critical for making decisions about its suitability and validity for research purposes.

Objective

To examine data source agreement of myocardial infarction (MI) diagnoses recorded in CPRD Aurum compared with linked Hospital Episode Statistics (HES) data. This comparison provides information on CPRD Aurum data correctness (accuracy, validity) and completeness (presence, missingness).

Methods

Patients with MI diagnoses recorded in either data source were selected from a random sample of 50,000 patients in CPRD Aurum with HES linkage (1997–2017). Correctness was defined as the proportion of MI cases in CPRD Aurum with a concordant MI diagnosis recorded in HES or with strong supporting evidence in either data source. Completeness was defined as the proportion of patients with primary HES-coded MIs with strong supporting evidence that were also present in CPRD Aurum.

Results

There were 1260 patients with MI recorded in the CPRD Aurum sample. The overall correctness of the recorded MI diagnoses was 94%: 986 patients (78%) had concordant diagnoses in HES within 90 days; 123 (10%) were concordant with HES, but with an inconclusive date and another 71 (6%) had strong supporting evidence for being a true MI case. There were 1125 patients with MI recorded in HES primary diagnosis fields with strong supporting evidence in either data source. Of these, 880 (78%) were present in CPRD Aurum, with completeness somewhat higher in more recent years.

Conclusion

MI diagnoses recorded in CPRD Aurum were highly likely to be correct, supporting its use in clinical research studies. Completeness was lower, indicating the need for data linkage for some studies.

Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience

OBJECTIVE

Large clinical databases are increasingly used for research and quality improvement. We describe an approach to data quality assessment from the General Medicine Inpatient Initiative (GEMINI), which collects and standardizes administrative and clinical data from hospitals.

METHODS

The GEMINI database contained 245 559 patient admissions at 7 hospitals in Ontario, Canada from 2010 to 2017. We performed 7 computational data quality checks and iteratively re-extracted data from hospitals to correct problems. Thereafter, GEMINI data were compared to data that were manually abstracted from the hospital's electronic medical record for 23 419 selected data points on a sample of 7488 patients.

RESULTS

Computational checks flagged 103 potential data quality issues, which were either corrected or documented to inform future analysis. For example, we identified the inclusion of canceled radiology tests, a time shift of transfusion data, and mistakenly processing the chemical symbol for sodium ("Na") as a missing value. Manual validation identified 1 important data quality issue that was not detected by computational checks: transfusion dates and times at 1 site were unreliable. Apart from that single issue, across all data tables, GEMINI data had high overall accuracy (ranging from 98%-100%), sensitivity (95%-100%), specificity (99%-100%), positive predictive value (93%-100%), and negative predictive value (99%-100%) compared to the gold standard.

DISCUSSION AND CONCLUSION

Computational data quality checks with iterative re-extraction facilitated reliable data collection from hospitals but missed 1 critical quality issue. Combining computational and manual approaches may be optimal for assessing the quality of large multisite clinical databases.

The mwtab Python Library for RESTful Access and Enhanced Quality Control, Deposition, and Curation of the Metabolomics Workbench Data Repository

The Metabolomics Workbench (MW) is a public scientific data repository consisting of experimental data and metadata from metabolomics studies collected with mass spectroscopy (MS) and nuclear magnetic resonance (NMR) analyses. MW has been constantly evolving; updating its ‘mwTab’ text file format, adding a JavaScript Object Notation (JSON) file format, implementing a REpresentational State Transfer (REST) interface, and nearly quadrupling the number of datasets hosted on the repository within the last three years. In order to keep up with the quickly evolving state of the MW repository, the ‘mwtab’ Python library and package have been continuously updated to mirror the changes in the ‘mwTab’ and JSONized formats and contain many new enhancements including methods for interacting with the MW REST interface, enhanced format validation features, and advanced features for parsing and searching for specific metabolite data and metadata. We used the enhanced format validation features to evaluate all available datasets in MW to facilitate improved curation and FAIRness of the repository. The ‘mwtab’ Python package is now officially released as version 1.0.1 and is freely available on GitHub and the Python Package Index (PyPI) under a Clear Berkeley Software Distribution (BSD) license with documentation available on ReadTheDocs.

Optimized Sample Preparation and Data Processing of Data-Independent Acquisition Methods for the Robust Quantification of Trace-Level Host Cell Protein Impurities in Antibody Drug Products

Host cell proteins (HCPs) are a major class of bioprocess-related impurities generated by the host organism and are generally present at low levels in purified biopharmaceutical products. The monitoring of these impurities is identified as an important critical quality attribute of monoclonal antibody (mAb) formulations not only due to the potential risk for the product stability and efficacy but also concerns linked to the immunogenicity of some of them. While overall HCP levels are usually monitored by enzyme-linked immunosorbent assay (ELISA), mass spectrometry (MS)-based approaches have been emerging as powerful and promising alternatives providing qualitative and quantitative information. However, a major challenge for liquid chromatography (LC)-MS-based methods is to deal with the wide dynamic range of drug products and the extreme sensitivity required to detect trace-level HCPs. In this study, we developed powerful and reproducible MS-based analytical workflows coupling optimized and efficient sample preparations, the library-free data-independent acquisition (DIA) method, and stringent validation criteria. The performances of several preparation protocols and DIA versus classical data-dependent acquisition (DDA) were evaluated using a series of four commercially available drug products. Depending on the selected protocols, the user has access to different information: on the one hand, a deep profiling of tens of identified HCPs and on the other hand, accurate and reproducible (coefficients of variation (CVs) < 12%) quantification of major HCPs. Overall, a final global HCP amount of a few tens of ng/mg mAb in these mAb samples was measured, while reaching a sensitivity down to the sub-ng/mg mAb level. Thus, this straightforward and robust approach can be intended as a routine quality control for any drug product analysis.

Quality and completeness of diagnoses recorded in the new CPRD Aurum Database: evaluation of pulmonary embolism

PURPOSE

The Clinical Practice Research Datalink (CPRD) now provides a new medical record database, CPRD Aurum. This is the first of several studies being undertaken to assess the quality and completeness of CPRD Aurum data for research endeavors.

METHODS

We identified patients with a pulmonary embolism (PE) diagnosis from a random sample of 50 000 patients in CPRD Aurum and compared the diagnoses using data from Hospital Episode Statistics (HES). We calculated the proportion of PE cases recorded in CPRD Aurum who also had a PE diagnosis recorded in HES. We also evaluated completeness by identifying all PE diagnoses in HES and calculating the proportion also present in CPRD Aurum.

RESULTS

The study included 781 PE patients: 580 had a PE in CPRD Aurum, 632 had a PE in HES, and 431 had a PE in both. The proportion of patients with anticoagulated PE in CPRD Aurum confirmed by HES was 76.8%. The completeness of primary hospitalized PE HES events compared to CPRD Aurum was 79.1%. In most instances, there was a plausible explanation for the presence of a PE in only one of the two data sources.

CONCLUSIONS

The results of this study are reassuring and suggest that the correctness (eg, quality, accuracy) and completeness of diagnosis information in CPRD Aurum are promising with respect to serious acute conditions that require medical attention. Evaluation of other data elements will provide additional insight into this new data resource and its utility for medical research.

Validation of Low-Cost Sensors in Measuring Real-Time PM10 Concentrations at Two Sites in Delhi National Capital Region

In the present study, we assessed for the first time the performance of our custom-designed low-cost Particulate Matter (PM) monitoring devices (Atmos) in measuring PM10 concentrations. We examined the ambient PM10 levels during an intense measurement campaign at two sites in the Delhi National Capital Region (NCR), India. In this study, we validated the un-calibrated Atmos for measuring ambient PM10 concentrations at highly polluted monitoring sites. PM10 concentration from Atmos, containing laser scattering-based Plantower PM sensor, was comparable with that measured from research-grade scanning mobility particle sizers (SMPS) in combination with optical particle sizers (OPS) and aerodynamic particle sizers (APS). The un-calibrated sensors often provided accurate PM10 measurements, particularly in capturing real-time hourly concentrations variations. Quantile-Quantile plots (QQ-plots) for data collected during the selected deployment period showed positively skewed PM10 datasets. Strong Spearman's rank-order correlations (rs = 0.64-0.83) between the studied instruments indicated the utility of low-cost Plantower PM sensors in measuring PM10 in the real-world context. Additionally, the heat map for weekly datasets demonstrated high R2 values, establishing the efficacy of PM sensor in PM10 measurement in highly polluted environmental conditions.

The role of multiple observations in small-molecule single-crystal service X-ray structure determination

In order to gain a better understanding of how to improve the quality of small-molecule single-crystal X-ray diffraction data achievable in a finite time, a study was carried out to investigate the effect of varying the multiplicity, acquisition time, detector binning, maximum resolution and completeness. The results suggest that, unless there are strong arguments for a different strategy, a good routine procedure might be to optimize the conditions necessary to get the best data from single scans, and then choose a multiplicity of observations (MoO) to utilize the available time fully. Different strategies may be required if the crystal is highly absorbing, is larger than the incident beam, is enclosed in a capillary tube or is unusual in some other way. The signal-to-noise ratio should be used with care, as collecting data for longer or at higher multiplicity appears to give a systematic underestimate of the intensity uncertainties. Further, the results demonstrate that including poor-quality data in a refinement may degrade the result and, in the general case, the accidental omission of reflections has a very small impact on the refinement as long as they are omitted at random. Systematic omission of reflections needs a convincing procedural justification.

Validation of Sentinel-3A/3B Satellite Altimetry Wave Heights with Buoy and Jason-3 Data

The validation of significant wave height (SWH) data measured by the Sentinel-3A/3B SAR Altimeter (SRAL) is essential for the application of the data in ocean wave monitoring, forecasting and wave climate studies. Sentinel-3A/3B SWH data are validated by comparisons with U. S. National Data Buoy Center (NDBC) buoys, using a spatial scale of 25 km and a temporal scale of 30 min, and with Jason-3 data at their crossovers, using a time difference of less than 30 min. The comparisons with NDBC buoy data show that the root-mean-square error (RMSE) of Sentinel-3A SWH is 0.30 m, and that of Sentinel-3B is no more than 0.31 m. The pseudo-Low-Resolution Mode (PLRM) SWH is slightly better than that of the Synthetic Aperture Radar (SAR) mode. The statistical analysis of Sentinel-3A/3B SWH in the bin of 0.5 m wave height shows that the accuracy of Sentinel-3A/3B SWH data decreases with increasing wave height. The analysis of the monthly biases and RMSEs of Sentinel-3A SWH shows that Sentinel-3A SWH are stable and have a slight upward trend with time. The comparisons with Jason-3 data show that SWH of Sentinel-3A and Jason-3 are consistent in the global ocean. Finally, the piecewise calibration functions are given for the calibration of Sentinel-3A/3B SWH. The results of the study show that Sentinel-3A/3B SWH data have high accuracy and remain stable.

Validating peritonsillar abscess drainage rates using the Pediatric hospital information system data

OBJECTIVES

To evaluate the accuracy of the Children's Hospital Association's Pediatric Health Information System (PHIS) registry data in determining surgical drainage rates as compared to a chart review on the same cohort of children with diagnosis of peritonsillar abscess.

STUDY DESIGN

Retrospective analysis.

METHODS

Our analysis included 200 children, ages 2 to 17 years, treated for a peritonsillar abscess from 2011 to 2016. The primary outcome was to determine the sensitivity, specificity, predictive values, receiver operating characteristics (ROC), and likelihood ratios of surgical drainage rates comparing the PHIS database to manual chart review of the same patients.

RESULTS

One hundred and fifteen (58%) children underwent drainage by chart review, whereas 87 (44%) had a drainage procedure by PHIS data. Age was a significant predictor of abscess drainage by chart review (age coefficient = 0.10; standard error = 0.04; 2 = 5.8; P = 0.02; odds ratio = 1.1; 95% confidence interval [CI] = 1.01-1.19). When using the chart review as the reference value for surgical drainage, the PHIS data had a sensitivity of 76% and 100% specificity. The positive and negative predictive values were 100% and 75%, respectively. The ROC area was 0.88 (95% CI, 84 to 92). Cases that lacked a clear procedure note composed the false negative cases in the PHIS.

CONCLUSION

At our institution, the PHIS Administrative data was adequate at predicting surgical drainage of the peritonsillar abscess when compared to chart review.

LEVEL OF EVIDENCE

4 Laryngoscope, 130:238-241, 2020.

A Review of Clinical Data Management Systems Used in Clinical Trials

BACKGROUND

A clinical data management system is a software supporting the data management process in clinical trials. In this system, the effective support of clinical data management dimensions leads to the increased accuracy of results and prevention of diversion in clinical trials. The aim of this review article was to investigate the dimensions of data management in clinical data management systems.

METHODS

This study was conducted in 2017. The used databases included Web of Science, Scopus, Science Direct, ProQuest, Ovid Medline and PubMed. The search was conducted over a period of 10 years from 2007 to 2017. The initial number of studies was 101 reaching 19 in the final stage. The final studies were described and compared in terms of the year, country and dimensions of the clinical data management process in clinical trials.

RESULTS

The research findings indicated that none of the systems completely supported the data management dimensions in clinical trials. Although these systems were developed for supporting the clinical data management process, they were similar to electronic data capture systems in many cases. The most significant dimensions of data management in such systems were data collection or entry, report, validation, and security maintenance.

CONCLUSION

Seemingly, not sufficient attention has been paid to automate all dimensions of the clinical data management process in clinical trials. However, these systems could take positive steps towards changing the manual processes of clinical data management to electronic processes.

How to quantify animal activity from radio-frequency identification (RFID) recordings

Automated animal monitoring via radio-frequency identification (RFID) technology allows efficient and extensive data sampling of individual activity levels and is therefore commonly used for ecological research. However, processing RFID data is still a largely unresolved problem, which potentially leads to inaccurate estimates for behavioral activity. One of the major challenges during data processing is to isolate independent behavioral actions from a set of superfluous, nonindependent detections. As a case study, individual blue tits (Cyanistes caeruleus) were simultaneously monitored during reproduction with both video recordings and RFID technology. We demonstrated how RFID data can be processed based on the time spent in- and outside a nest box. We then validated the number and timing of nest visits obtained from the processed RFID dataset by calibration against video recordings. The video observations revealed a limited overlap between the time spent in- and outside the nest box, with the least overlap at 23 s for both sexes. We then isolated exact arrival times from redundant RFID registrations by erasing all successive registrations within 23 s after the preceding registration. After aligning the processed RFID data with the corresponding video recordings, we observed a high accuracy in three behavioral estimates of parental care (individual nest visit rates, within-pair alternation and synchronization of nest visits). We provide a clear guideline for future studies that aim to implement RFID technology in their research. We argue that our suggested RFID data processing procedure improves the precision of behavioral estimates, despite some inevitable drawbacks inherent to the technology. Our method is useful, not only for other cavity breeding birds, but for a wide range of (in)vertebrate species that are large enough to be fitted with a tag and that regularly pass near or through a fixed antenna.

Breastfeeding, Infant Formula, and Introduction to Complementary Foods-Comparing Data Obtained by Questionnaires and Health Visitors' Reports to Weekly Short Message Service Text Messages

BACKGROUND

Studies on prevalence and effects of breastfeeding call for reliable and precise data collection to optimize infant nutrition, growth, and health. Data on breastfeeding and infant nutrition are at risk of, for example, recall bias or social desirability bias.

OBJECTIVE

The aim of the present analysis was to compare data on infant nutrition, that is, breastfeeding, use of infant formula, and introduction to complementary foods, obtained by four different methods. We assumed that weekly short message service (SMS) questions were the most reliable method, to which the other methods were compared.

DESIGN

The study population was part of the Odense Child Cohort. The four methods used were: (a) self-administered questionnaire 3 months postpartum, (b) self-administered questionnaire 18 months postpartum, (c) registrations from health visitors visiting the families several times within the first year of life, and (d) weekly SMS questions introduced shortly after birth.

RESULTS

In total, 639 singleton mothers with data from all four methods were included. The proportion of mothers initiating breastfeeding varied from 86% to 97%, the mean duration of exclusive breastfeeding from 12 to 19 weeks, and the mean age when introduced to complementary foods from 19 to 21 weeks. The mean duration of any breastfeeding was 33 weeks across methods.

CONCLUSIONS

Compared with the weekly SMS questions, the self-administered questionnaires and the health visitors' reports resulted in a greater proportion of mothers with an unknown breastfeeding status, a longer duration of exclusive breastfeeding and later introduction to complementary foods, while the duration of any breastfeeding did not differ.

Quality control and data validation procedure in large-scale quantitative monitoring of mosquito density: the case of Aedes albopictus in Emilia-Romagna region, Italy

The risk of local transmission of chikungunya, dengue, and Zika viruses from viremic travelers arriving from affected areas is a real possibility in southern Europe, due to the large presence of the vector Aedes albopictus. The first chikungunya outbreak in Europe occurred in the Emilia-Romagna region, Italy, in 2007, prompting the regional public health department to implement an Ae. albopictus monitoring and control plan. This paper presents the procedure developed for the quality control of the data collected by the regional ovitrap monitoring network. The quality control procedure consists of four successive steps; the first step is data acquisition and the second is an automatic data processing step, while skilled technicians according to specific procedures perform the third and fourth steps by checking the exact position of the ovitraps and by conducting human landing collections, respectively. The paper also presents the results of this quality control procedure applied to the data collected through the large-scale Ae. albopictus monitoring operation performed in Emilia-Romagna in 2015.

A new marker based on the avian spindlin gene that is able to sex most birds, including species problematic to sex with CHD markers

We have developed a new marker (Z43B) that can be successfully used to identify the sex of most birds (69%), including species difficult or impossible to sex with other markers. We utilized the zebra finch Taeniopygia guttata EST microsatellite sequence (CK309496) which displays sequence homology to the 5' untranslated region (UTR) of the avian spindlin gene. This gene is known to be present on the Z and W chromosomes. To maximize cross-species utility, the primer set was designed from a consensus sequence created from homologs of CK309496 that were isolated from multiple distantly related species. Both the forward and reverse primer sequences were 100% identical to 14 avian species, including the Z chromosome of eight species and the chicken Gallus gallus W chromosome, as well as the saltwater crocodile Crocodylus porosus. The Z43B primer set was assessed by genotyping individuals of known sex belonging to 61 non-ratite species and a single ratite. The Z and W amplicons differed in size making it possible to distinguish between males (ZZ) and females (ZW) for the majority (69%) of non-ratite species tested, comprising 10 orders of birds. We predict that this marker will be useful for obtaining sex-typing data for ca 6,869 species of birds (69% of non-ratites but not galliforms). A wide range of species could be sex-typed including passerines, shorebirds, eagles, falcons, bee-eaters, cranes, shags, parrots, penguins, ducks, and a ratite species, the brown kiwi, Apteryx australis. Those species sexed include species impossible or problematic to sex-type with other markers (magpie, albatross, petrel, eagle, falcon, crane, and penguin species). Zoo Biol. 35:533-545, 2016. © 2016 The Authors. Zoo Biology published by Wiley Periodicals, Inc.

A generalized approach for producing, quantifying, and validating citizen science data from wildlife images

Citizen science has the potential to expand the scope and scale of research in ecology and conservation, but many professional researchers remain skeptical of data produced by nonexperts. We devised an approach for producing accurate, reliable data from untrained, nonexpert volunteers. On the citizen science website www.snapshotserengeti.org, more than 28,000 volunteers classified 1.51 million images taken in a large-scale camera-trap survey in Serengeti National Park, Tanzania. Each image was circulated to, on average, 27 volunteers, and their classifications were aggregated using a simple plurality algorithm. We validated the aggregated answers against a data set of 3829 images verified by experts and calculated 3 certainty metrics-level of agreement among classifications (evenness), fraction of classifications supporting the aggregated answer (fraction support), and fraction of classifiers who reported "nothing here" for an image that was ultimately classified as containing an animal (fraction blank)-to measure confidence that an aggregated answer was correct. Overall, aggregated volunteer answers agreed with the expert-verified data on 98% of images, but accuracy differed by species commonness such that rare species had higher rates of false positives and false negatives. Easily calculated analysis of variance and post-hoc Tukey tests indicated that the certainty metrics were significant indicators of whether each image was correctly classified or classifiable. Thus, the certainty metrics can be used to identify images for expert review. Bootstrapping analyses further indicated that 90% of images were correctly classified with just 5 volunteers per image. Species classifications based on the plurality vote of multiple citizen scientists can provide a reliable foundation for large-scale monitoring of African wildlife.

Citizen science networks in natural history and the collective validation of biodiversity data

Biodiversity data are in increasing demand to inform policy and management. A substantial portion of these data is generated in citizen science networks. To ensure the quality of biodiversity data, standards and criteria for validation have been put in place. We used interviews and document analysis from the United Kingdom and The Netherlands to examine how data validation serves as a point of connection between the diverse people and practices in natural history citizen science networks. We found that rather than a unidirectional imposition of standards, validation was performed collectively. Specifically, it was enacted in ongoing circulations of biodiversity records between recorders and validators as they jointly negotiated the biodiversity that was observed and the validity of the records. These collective validation practices contributed to the citizen science character or natural history networks and tied these networks together. However, when biodiversity records were included in biodiversity-information initiatives on different policy levels and scales, the circulation of records diminished. These initiatives took on a more extractive mode of data use. Validation ceased to be collective with important consequences for the natural history networks involved and citizen science more generally.

A rational approach to legacy data validation when transitioning between electronic health record systems

OBJECTIVE

The objective of this project was to use statistical techniques to determine the completeness and accuracy of data migrated during electronic health record conversion.

METHODS

Data validation during migration consists of mapped record testing and validation of a sample of the data for completeness and accuracy. We statistically determined a randomized sample size for each data type based on the desired confidence level and error limits.

RESULTS

The only error identified in the post go-live period was a failure to migrate some clinical notes, which was unrelated to the validation process. No errors in the migrated data were found during the 12- month post-implementation period.

CONCLUSIONS

Compared to the typical industry approach, we have demonstrated that a statistical approach to sampling size for data validation can ensure consistent confidence levels while maximizing efficiency of the validation process during a major electronic health record conversion.

Efficient, quality-assured data capture in operational research through innovative use of open-access technology

Ensuring quality of data during electronic data capture has been one of the most neglected components of operational research. Multicentre studies are also challenged with issues about logistics of travel, training, supervision, monitoring and troubleshooting support. Allocating resources to these issues can pose a significant bottleneck for operational research in resource-limited settings. In this article, we describe an innovative and efficient way of coordinating data capture in multicentre operational research using a combination of three open access technologies-EpiData for data capture, Dropbox for sharing files and TeamViewer for providing remote support.

Genotyping of CYP2C19 polymorphisms and its clinical validation in the ethnic Arab population

OBJECTIVES

The drug-metabolizing enzymes and transporters (DMET) Plus microarray and x-Tag assays have recently been developed for genotyping individuals in personalized medicine. Furthermore, the cytochrome 450-2C19 (CYP2C19) is a key metabolic enzyme encoded by a polymorphic gene commonly associated with diminished metabolism and variable clinical responses to several drugs in an ethnicity-dependent fashion. Therefore, validation of these clinical procedures as well as knowledge of the ethnic-specific incidences of these gene variants is prerequisite for determining their clinical relevance in any given population.

METHODS

We determined the distribution of familiar CYP2C19 variants by the DMET Plus chip in 600 candidates and replicated the findings by the Affymetrix Axiom Genome-Wide Asian Structure Identification Array in 5413 individuals, all Saudis of ethic Arab origin. We then tested the robustness of employing the Luminex xMAP system clinically by comparing the results of genotyping 500 Saudi individuals visiting the Blood Bank of our institution with the findings of the two platforms.

KEY FINDINGS

The DMET Plus genotyping revealed that eight of the CYP2C19 variants showed some changes. Thereby, the CYP2C19*17 exhibited the highest minor allele frequency (MAF) of 0.256, followed by the CYP2C19_801 (frequency = 0.055). Six other variants, including the CYP2C19*3, showed MAF in the range of 0.001-0.002. We replicated the frequencies of the CYP2C19*17 and CYP2C19*3, and additionally established that of the CYP2C19*2 (0.099) using the Axiom platform. The xTag genotyping also indicated that 0.834 of the 500 Saudi individuals were extensive metabolizers (*1/*1), 0.158 carried the *1/*2 genotype, 0.01% carried *2/*2 (poor metabolizers) and one each (0.2%) harboured the *1/*8, *2/*3 (intermediate metabolizers) and *8/*8 (poor metabolizers) genotypes.

CONCLUSIONS

The results showed reproducible genotyping of the CYP2C19 variants in the Saudi Arab population using two Affymetrix platforms and phenotyping using the Luminex xTag assay. The prevalence of two clinically relevant genotypes (CYP2C19*2 and CYP2C19*3) were similar to other ethnic groups, while that of the CYP2C19*17 was comparably higher.

How should forensic anthropologists correct national weather service temperature data for use in estimating the postmortem interval?

This study examines the correlation between site-specific and retrospectively collected temperature data from the National Weather Service (NWS) over an extended time period. Using iButtonLink thermochrons (model DS1921G), hourly temperature readings were collected at 15 sites (1 validation; 14 experimental) from December 2010 to January 2012. Comparison between the site-specific temperature data and data retrieved from an official reporter of NWS temperature data shows statistically significant differences between the two in 71.4% (10/14) of cases. The difference ranged between 0.04 and 2.81°C. Examination of both regression and simple adjustment of the mean difference over extended periods (1, 2, 3, 4, 5, 6, & 9 months) suggests that on the timescale typical in forensic anthropology cases neither method of correction is consistent or reliable and that forensic anthropologists would be better suited using uncorrected NWS temperature data when the postmortem interval is extended.

Quality validation of data in national haemovigilance systems in Europe: report of a survey on current state of practice

European Union member states must have national haemovigilance reporting of serious adverse reactions and events. We sent national competent authorities an email questionnaire about data validation. Responses were received from 23/27 countries. Nine previously had no national haemovigilance system. In 13 (57%), the serious adverse reactions and events can be verified. Coverage of blood establishments is documented in 20 systems (87%) and of hospitals in 15 systems (65%). Although all member states have implemented haemovigilance systems, there are currently wide variations in data quality assurance, not allowing comparisons between countries.