A comparison of smartphones to paper-based questionnaires for routine influenza sentinel surveillance, Kenya, 2011-2012

Advance Warning and Response Systems in Kenya: A Scoping Review

Introduction

Infectious diseases (IDs) cause approximately 13.7 million deaths globally. The Kenyan Advance Warning and Response Systems (AW&RS) against ID outbreaks is a core capacity of the 2005 International Health Regulations and a key indicator of health security. We mapped evidence on Kenya's AW&RS and their enablers, and barriers for successfully detecting IDs, including climate-sensitive IDs.

Methods

We searched Cochrane Library, MEDLINE, EMBASE, Web of Science, Africa Index Medicus, and SCOPUS before August 26th, 2024. We also searched for grey literature on the Google Scholar search engine alongside the main repositories of Kenyan Universities. Two independent reviewers conducted study selection, while one reviewer extracted data. Discrepancies were resolved through discussion. Results were synthesised narratively and thematically.

Results

The search yielded 4,379 records from databases and 1,363 articles from websites, university repositories, and citations; we included 166 articles in the analysis. Integrated Disease Surveillance and Response (IDSR) and cohort surveillance systems were the most common (37.2%). Most studies were concentrated in Nairobi County (25.7%) and reported on malaria (23.6%). Most systems (82.4%) monitored the disease burden and outbreaks using hospital-based data (35.1%) and automated alert mechanisms (27.7%). National bulletins report a temporal association between environmental factors and disease prevalence. Malaria, Rift Valley Fever (RVF), and cholera cases increased with higher precipitation, lower temperatures and increased vegetative index. AW&RS used the accuracy and reliability of the model prediction to measure the system's performance. Effectiveness was evaluated based on system acceptability and timeliness. Health system factors were predominant, with 121 enablers and 127 barriers. Key enablers included skilled personnel (13 studies), whereas inadequate finances were a major barrier (21 studies).

Conclusion

Most AW&RS were IDSR and cohort-based surveillance. Climate changes have resulted in observed trends in diseases such as malaria and RVF, but further studies are needed to determine causal links. Insufficient funding hinders the effective implementation of AW&RS. Future research should assess the cost drivers influencing system effectiveness.

Time Efficiency, Reliability, and User Satisfaction of the Tooth Memo App for Recording Oral Health Information: Cross-Sectional Questionnaire Study

BACKGROUND

Digitalizing oral health data through an app can help manage the extensive data obtained through oral health surveys. The Tooth Memo app collects data from oral health surveys and personal health information.

OBJECTIVE

This study aims to evaluate the evaluate the time efficiency, reliability, and user satisfaction of the Tooth Memo app.

METHODS

There are 2 sections in the Tooth Memo app: oral health survey and personal oral health record. For the oral health survey section of the Tooth Memo app, different data entry methods were compared and user satisfaction was evaluated. Fifth-year dental students had access to the oral health survey section in the Tooth Memo app during their clinical work. The time required for data entry, analysis, and summary of oral health survey data by 3 methods, that is, pen-and-paper (manual), Tooth Memo app on iOS device, and Tooth Memo app on Android device were compared among 3 data recorders who entered patients' information on decayed, missing, and filled permanent teeth (DMFT) index and community periodontal index (CPI), which were read aloud from the database of 103 patients by another dental personnel. The interobserver reliability of the 3 different data-entering procedures was evaluated by percent disagreement and kappa statistic values. Laypeople had access to the personal oral health record section of this app, and their satisfaction was evaluated through a Likert scale questionnaire. The satisfaction assessments for both sections of the Tooth Memo app involved the same set of questions on the app design, usage, and overall satisfaction.

RESULTS

Of the 103 dental records on DMFT and CPI, 5.2% (177/3399) data points were missing in the manual data entries, but no data on tooth status were missing in the Android and iOS methods. Complete CPI information was provided by all 3 methods. Transferring data from paper to computer took an average of 55 seconds per case. The manual method required 182 minutes more than the iOS or Android methods to clean the missing data and transfer and analyze the tooth status data of 103 patients. The users, that is, 109 fifth-year dental students and 134 laypeople, expressed high satisfaction with using the Tooth Memo app. The overall satisfaction with the oral health survey ranged between 3 and 10, with an average (SD) of 7.86 (1.46). The overall satisfaction with the personal oral health record ranged between 4 and 10, with an average (SD) of 8.09 (1.28).

CONCLUSIONS

The Tooth Memo app was more efficacious than manual data entry for collecting data of oral health surveys. Dental personnel as well as general users reported high satisfaction when using this app.

Adaptation of a Danish online version of the Oxford Physical Activity Questionnaire (OPAQ) for secondary school students—a pilot study

Objective

To adapt and partly validate a Danish online version of the patient-reported outcome measure (PROM) Oxford Physical Activity Questionnaire (“OPAQ”) and evaluate mobile phones and tablets as data capturing tool to identify potential problems and deficiencies in the PROM prior to implementation in the full study.

Methods

The OPAQ was translated into Danish by a formalised forward-backward translation procedure. Face validity was examined by interviewing 12 school students aged 10–15, recruited from two Danish public schools. After modifications, the online version of the Danish OPAQ was pilot tested in a convenience sample of seven school students for 1 week. Simultaneous objective accelerometer data were captured during the registration period.

Results

No major challenges were identified when translating OPAQ. Based on the interviews, the Danish version of OPAQ was perceived to be easy to understand in general, and the questions were relevant for tracking activities during the week. Five of the 12 participants had difficulties with understanding the introductory question: “what is your cultural background” in the original OPAQ. The interviews revealed that the participants recalling 7 days forgot to record some of the physical activity they had done during the week, indicating issues with the weekly recall method.

After transforming to the online version, this was reported to be easy and quick to fill in (taking 1–3 min per day), and participants reported the daily design was helpful to remember activities. There was good correspondence between the online version and objective actigraphs with a tendency to underreport. Six participants reported 10–60 min less moderate to vigorous physical activity compared to the actigraphs, while one participant reported 3 min more.

Conclusion

Participants found the online OPAQ quick and easy to complete during a 1-week period. Completing daily rather than weekly may help limit issues with recall. Overall, there was good agreement between the objective actigraphs and the OPAQ, though the OPAQ tended to slightly underreport moderate to vigorous physical activity.

The Danish online version of OPAQ may be useful for capturing school students’ physical activity when objective measures are not feasible.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-022-01108-x.

Factors Affecting the Transition from Paper to Digital Data Collection for Mobile Tuberculosis Active Case Finding in Low Internet Access Settings in Pakistan

Between September 2020 and March 2021, Mercy Corps piloted hybrid digital (CAPI) and paper-based (PAPI) data collection as part of its tuberculosis (TB) active case finding strategy. Data were collected using CAPI and PAPI at 140 TB chest camps in low Internet access areas of Punjab and Khyber Pakhtunkhwa provinces in Pakistan. PAPI data collection was performed primarily during the camp and entered using a tailor-performed CAPI tool after camps. To assess the feasibility of this hybrid approach, quality of digital records were measured against the paper “gold standard”, and user acceptance was evaluated through focus group discussions. Completeness of digital data varied by indicator, van screening team, and month of implementation: chest camp attendees and pulmonary TB cases showed the highest CAPI/PAPI completeness ratios (1.01 and 0.96 respectively), and among them, all forms of TB diagnosis and treatment initiation were lowest (0.63 and 0.64 respectively). Vans entering CAPI data with high levels of completeness generally did so for all indicators, and significant differences in mean indicator completeness rates between PAPI and CAPI were observed between vans. User feedback suggested that although the CAPI tool required practice to gain proficiency, the technology was appreciated and will be better perceived once double entry in CAPI and PAPI can transition to CAPI only. CAPI data collection enables data to be entered in a more timely fashion in low-Internet-access settings, which will enable more rapid, evidence-based program steering. The current system in which double data entry is conducted to ensure data quality is an added burden for staff with many activities. Transitioning to a fully digital data collection system for TB case finding in low-Internet-access settings requires substantial investments in M&E support, shifts in data reporting accountability, and technology to link records of patients who pass through separate data collection stages during chest camp events.

Going digital: added value of electronic data collection in 2018 Afghanistan Health Survey

BACKGROUND

Through a nationally representative household survey in Afghanistan, we conducted an operational study in two relatively secure provinces comparing effectiveness of computer-aided personal interviewing (CAPI) with paper-and-pencil interviewing (PAPI).

METHODS

In Panjshir and Parwan provinces, household survey data were collected using paper questionnaires in 15 clusters, and OpenDataKit (ODK) software on electronic tablets in 15 other clusters. Added value was evaluated from three perspectives: efficient implementation, data quality, and acceptability. Efficiency was measured through financial expenditures and time stamped data. Data quality was measured by examining completeness. Acceptability was studied through focus group discussions with survey staff.

RESULTS

Survey costs were 68% more expensive in CAPI clusters compared to PAPI clusters, due primarily to the upfront one-time investment for survey programming. Enumerators spent significantly less time administering surveys in CAPI cluster households (248 min survey time) compared to PAPI (289 min), for an average savings of 41 min per household (95% CI 25-55). CAPI offered a savings of 87 days for data management over PAPI. Among 49 tracer variables (meaning responses were required from all respondents), small differences were observed between PAPI and CAPI. 2.2% of the cleaned dataset's tracer data points were missing in CAPI surveys (1216/ 56,073 data points), compared to 3.2% in PAPI surveys (1953/ 60,675 data points). In pre-cleaned datasets, 3.9% of tracer data points were missing in CAPI surveys (2151/ 55,092 data points) compared to 3.2% in PAPI surveys (1924/ 60,113 data points). Enumerators from Panjsher and Parwan preferred CAPI over PAPI due to time savings, user-friendliness, improved data security, and less conspicuity when traveling; however approximately half of enumerators trained from all 34 provinces reported feeling unsafe due to Taliban presence. Community and household respondent skepticism could be resolved by enumerator reassurance. Enumerators shared that in the future, they prefer collecting data using CAPI when possible.

CONCLUSIONS

CAPI offers clear gains in efficiency over PAPI for data collection and management time, although costs are relatively comparable even without the programming investment. However, serious field staff concerns around Taliban threats and general insecurity mean that CAPI should only be conducted in relatively secure areas.

Electronic data collection, management and analysis tools used for outbreak response in low- and middle-income countries: a systematic review and stakeholder survey

BACKGROUND

Use of electronic data collection, management and analysis tools to support outbreak response is limited, especially in low income countries. This can hamper timely decision-making during outbreak response. Identifying available tools and assessing their functions in the context of outbreak response would support appropriate selection and use, and likely more timely data-driven decision-making during outbreaks.

METHODS

We conducted a systematic review and a stakeholder survey of the Global Outbreak Alert and Response Network and other partners to identify and describe the use of, and technical characteristics of, electronic data tools used for outbreak response in low- and middle-income countries. Databases included were MEDLINE, EMBASE, Global Health, Web of Science and CINAHL with publications related to tools for outbreak response included from January 2010-May 2020. Software tool websites of identified tools were also reviewed. Inclusion and exclusion criteria were applied and counts, and proportions of data obtained from the review or stakeholder survey were calculated.

RESULTS

We identified 75 electronic tools including for data collection (33/75), management (13/75) and analysis (49/75) based on data from the review and survey. Twenty-eight tools integrated all three functionalities upon collection of additional information from the tool developer websites. The majority were open source, capable of offline data collection and data visualisation. EpiInfo, KoBoCollect and Open Data Kit had the broadest use, including for health promotion, infection prevention and control, and surveillance data capture. Survey participants highlighted harmonisation of data tools as a key challenge in outbreaks and the need for preparedness through training front-line responders on data tools. In partnership with the Global Health Network, we created an online interactive decision-making tool using data derived from the survey and review.

CONCLUSIONS

Many electronic tools are available for data -collection, -management and -analysis in outbreak response, but appropriate tool selection depends on knowledge of tools' functionalities and capabilities. The online decision-making tool created to assist selection of the most appropriate tool(s) for outbreak response helps by matching requirements with functionality. Applying the tool together with harmonisation of data formats, and training of front-line responders outside of epidemic periods can support more timely data-driven decision making in outbreaks.

Integrating Behavior of Children with Profound Intellectual, Multiple, or Severe Motor Disabilities With Location and Environment Data Sensors for Independent Communication and Mobility: App Development and Pilot Testing

Background

Children with profound intellectual and multiple disabilities (PIMD) or severe motor and intellectual disabilities (SMID) only communicate through movements, vocalizations, body postures, muscle tensions, or facial expressions on a pre- or protosymbolic level. Yet, to the best of our knowledge, there are few systems developed to specifically aid in categorizing and interpreting behaviors of children with PIMD or SMID to facilitate independent communication and mobility. Further, environmental data such as weather variables were found to have associations with human affects and behaviors among typically developing children; however, studies involving children with neurological functioning impairments that affect communication or those who have physical and/or motor disabilities are unexpectedly scarce.

Objective

This paper describes the design and development of the ChildSIDE app, which collects and transmits data associated with children’s behaviors, and linked location and environment information collected from data sources (GPS, iBeacon device, ALPS Sensor, and OpenWeatherMap application programming interface [API]) to the database. The aims of this study were to measure and compare the server/API performance of the app in detecting and transmitting environment data from the data sources to the database, and to categorize the movements associated with each behavior data as the basis for future development and analyses.

Methods

This study utilized a cross-sectional observational design by performing multiple single-subject face-to-face and video-recorded sessions among purposively sampled child-caregiver dyads (children diagnosed with PIMD/SMID, or severe or profound intellectual disability and their primary caregivers) from September 2019 to February 2020. To measure the server/API performance of the app in detecting and transmitting data from data sources to the database, frequency distribution and percentages of 31 location and environment data parameters were computed and compared. To categorize which body parts or movements were involved in each behavior, the interrater agreement κ statistic was used.

Results

The study comprised 150 sessions involving 20 child-caregiver dyads. The app collected 371 individual behavior data, 327 of which had associated location and environment data from data collection sources. The analyses revealed that ChildSIDE had a server/API performance >93% in detecting and transmitting outdoor location (GPS) and environment data (ALPS sensors, OpenWeatherMap API), whereas the performance with iBeacon data was lower (82.3%). Behaviors were manifested mainly through hand (22.8%) and body movements (27.7%), and vocalizations (21.6%).

Conclusions

The ChildSIDE app is an effective tool in collecting the behavior data of children with PIMD/SMID. The app showed high server/API performance in detecting outdoor location and environment data from sensors and an online API to the database with a performance rate above 93%. The results of the analysis and categorization of behaviors suggest a need for a system that uses motion capture and trajectory analyses for developing machine- or deep-learning algorithms to predict the needs of children with PIMD/SMID in the future.

Digital Data Sources and Their Impact on People's Health: A Systematic Review of Systematic Reviews

Background: Digital data sources have become ubiquitous in modern culture in the era of digital technology but often tend to be under-researched because of restricted access to data sources due to fragmentation, privacy issues, or industry ownership, and the methodological complexity of demonstrating their measurable impact on human health. Even though new big data sources have shown unprecedented potential for disease diagnosis and outbreak detection, we need to investigate results in the existing literature to gain a comprehensive understanding of their impact on and benefits to human health. Objective: A systematic review of systematic reviews on identifying digital data sources and their impact area on people's health, including challenges, opportunities, and good practices. Methods: A multidatabase search was performed. Peer-reviewed papers published between January 2010 and November 2020 relevant to digital data sources on health were extracted, assessed, and reviewed. Results: The 64 reviews are covered by three domains, that is, universal health coverage (UHC), public health emergencies, and healthier populations, defined in WHO's General Programme of Work, 2019-2023, and the European Programme of Work, 2020-2025. In all three categories, social media platforms are the most popular digital data source, accounting for 47% (N = 8), 84% (N = 11), and 76% (N = 26) of studies, respectively. The second most utilized data source are electronic health records (EHRs) (N = 13), followed by websites (N = 7) and mass media (N = 5). In all three categories, the most studied impact of digital data sources is on prevention, management, and intervention of diseases (N = 40), and as a tool, there are also many studies (N = 10) on early warning systems for infectious diseases. However, they could also pose health hazards (N = 13), for instance, by exacerbating mental health issues and promoting smoking and drinking behavior among young people. Conclusions: The digital data sources presented are essential for collecting and mining information about human health. The key impact of social media, electronic health records, and websites is in the area of infectious diseases and early warning systems, and in the area of personal health, that is, on mental health and smoking and drinking prevention. However, further research is required to address privacy, trust, transparency, and interoperability to leverage the potential of data held in multiple datastores and systems. This study also identified the apparent gap in systematic reviews investigating the novel big data streams, Internet of Things (IoT) data streams, and sensor, mobile, and GPS data researched using artificial intelligence, complex network, and other computer science methods, as in this domain systematic reviews are not common.

MAES_GR: A Web-Based, Spatially Enabled Field Survey Platform for the MAES Implementation in Greece

This study presents a standardized approach to collecting, registering, and reporting field-survey data for baseline MAES (Mapping and Assessment of Ecosystems and their Services) information in Greece. This is accomplished through a web-based platform (MAES_GR) exclusively developed under the relevant, nation-wide LIFE-IP 4 NATURA project. Based on the European Commission’s guidance for ecosystem condition (EC) and ecosystem services (ES) MAES studies, we conceptualized and structured an online platform to support EC and ES assessments, integrating all relevant fields of information needed for registering EC and ES parameters. A novel algorithm calculating EC was also developed and it is available as an integral part of the platform. The use of the MAES_GR platform was evaluated during nationwide field surveys efforts, increasing time efficiency and reducing costs. Field recording of EC and ES pinpoint spatial priorities for ecosystem restoration, conservation and sustainable development. This work highlights that MAES implementation can be favored by the use of technology tools such as mobile survey platforms, developed according to scientific needs and policy guidelines. Such tools, apart from the data inventory phase, can be used for data analysis, synthesis and extraction, providing timely, standardized information suitable for reporting at the local, regional, national and European Union scale.

Data Quality and Cost-effectiveness Analyses of Electronic and Paper-Based Interviewer-Administered Public Health Surveys: Systematic Review

Background

A population-level survey (PLS) is an essential and standard method used in public health research that supports the quantification of sociodemographic events, public health policy development, and intervention designs. Data collection mechanisms in PLS seem to be a significant determinant in avoiding mistakes. Using electronic devices such as smartphones and tablet computers improves the quality and cost-effectiveness of public health surveys. However, there is a lack of systematic evidence to show the potential impact of electronic data collection tools on data quality and cost reduction in interviewer-administered surveys compared with the standard paper-based data collection system.

Objective

This systematic review aims to evaluate the impact of the interviewer-administered electronic data collection methods on data quality and cost reduction in PLS compared with traditional methods.

Methods

We conducted a systematic search of MEDLINE, CINAHL, PsycINFO, the Web of Science, EconLit, Cochrane CENTRAL, and CDSR to identify relevant studies from 2008 to 2018. We included randomized and nonrandomized studies that examined data quality and cost reduction outcomes, as well as usability, user experience, and usage parameters. In total, 2 independent authors screened the title and abstract, and extracted data from selected papers. A third author mediated any disagreements. The review authors used EndNote for deduplication and Rayyan for screening.

Results

Our search produced 3817 papers. After deduplication, we screened 2533 papers, and 14 fulfilled the inclusion criteria. None of the studies were randomized controlled trials; most had a quasi-experimental design, for example, comparative experimental evaluation studies nested on other ongoing cross-sectional surveys. A total of 4 comparative evaluations, 2 pre-post intervention comparative evaluations, 2 retrospective comparative evaluations, and 4 one-arm noncomparative studies were included. Meta-analysis was not possible because of the heterogeneity in study designs, types, study settings, and level of outcome measurements. Individual paper synthesis showed that electronic data collection systems provided good quality data and delivered faster compared with paper-based data collection systems. Only 2 studies linked cost and data quality outcomes to describe the cost-effectiveness of electronic data collection systems. Field data collectors reported that an electronic data collection system was a feasible, acceptable, and preferable tool for their work. Onsite data error prevention, fast data submission, and easy-to-handle devices were the comparative advantages offered by electronic data collection systems. Challenges during implementation included technical difficulties, accidental data loss, device theft, security concerns, power surges, and internet connection problems.

Conclusions

Although evidence exists of the comparative advantages of electronic data collection compared with paper-based methods, the included studies were not methodologically rigorous enough to combine. More rigorous studies are needed to compare paper and electronic data collection systems in public health surveys considering data quality, work efficiency, and cost reduction.

International Registered Report Identifier (IRRID)

RR2-10.2196/10678

Adherence of Mobile App-Based Surveys and Comparison With Traditional Surveys: eCohort Study

Background

eCohort studies offer an efficient approach for data collection. However, eCohort studies are challenged by volunteer bias and low adherence. We designed an eCohort embedded in the Framingham Heart Study (eFHS) to address these challenges and to compare the digital data to traditional data collection.

Objective

The aim of this study was to evaluate adherence of the eFHS app-based surveys deployed at baseline (time of enrollment in the eCohort) and every 3 months up to 1 year, and to compare baseline digital surveys with surveys collected at the research center.

Methods

We defined adherence rates as the proportion of participants who completed at least one survey at a given 3-month period and computed adherence rates for each 3-month period. To evaluate agreement, we compared several baseline measures obtained in the eFHS app survey to those obtained at the in-person research center exam using the concordance correlation coefficient (CCC).

Results

Among the 1948 eFHS participants (mean age 53, SD 9 years; 57% women), we found high adherence to baseline surveys (89%) and a decrease in adherence over time (58% at 3 months, 52% at 6 months, 41% at 9 months, and 40% at 12 months). eFHS participants who returned surveys were more likely to be women (adjusted odds ratio [aOR] 1.58, 95% CI 1.18-2.11) and less likely to be smokers (aOR 0.53, 95% CI 0.32-0.90). Compared to in-person exam data, we observed moderate agreement for baseline app-based surveys of the Physical Activity Index (mean difference 2.27, CCC=0.56), and high agreement for average drinks per week (mean difference 0.54, CCC=0.82) and depressive symptoms scores (mean difference 0.03, CCC=0.77).

Conclusions

We observed that eFHS participants had a high survey return at baseline and each 3-month survey period over the 12 months of follow up. We observed moderate to high agreement between digital and research center measures for several types of surveys, including physical activity, depressive symptoms, and alcohol use. Thus, this digital data collection mechanism is a promising tool to collect data related to cardiovascular disease and its risk factors.

Electronic data capture for large scale typhoid surveillance, household contact tracing, and health utilisation survey: Strategic Typhoid Alliance across Africa and Asia

Electronic data capture systems (EDCs) have the potential to achieve efficiency and quality in collection of multisite data. We quantify the volume, time, accuracy and costs of an EDC using large-scale census data from the STRATAA consortium, a comprehensive programme assessing population dynamics and epidemiology of typhoid fever in Malawi, Nepal and Bangladesh to inform vaccine and public health interventions. A census form was developed through a structured iterative process and implemented using Open Data Kit Collect running on Android-based tablets. Data were uploaded to Open Data Kit Aggregate, then auto-synced to MySQL-defined database nightly. Data were backed-up daily from three sites centrally, and auto-reported weekly. Pre-census materials' costs were estimated. Demographics of 308,348 individuals from 80,851 households were recorded within an average of 14.7 weeks range (13-16) using 65 fieldworkers. Overall, 21.7 errors (95% confidence interval: 21.4, 22.0) per 10,000 data points were found: 13.0 (95% confidence interval: 12.6, 13.5) and 24.5 (95% confidence interval: 24.1, 24.9) errors on numeric and text fields respectively. These values meet standard quality threshold of 50 errors per 10,000 data points. The EDC's total variable cost was estimated at US$13,791.82 per site. In conclusion, the EDC is robust, allowing for timely and high-volume accurate data collection, and could be adopted in similar epidemiological settings.

Comparative validity and reliability of the WeChat-based electronic and paper-and-pencil versions of the PISQ-12 for collecting participant-reported data in Chinese

Objective:

The objective of this study is to assess the consistency between the WeChat-based Pelvic Organ Prolapse/Urinary Incontinence Sexual Questionnaire short form (PISQ-12) in Chinese and the paper version and to determine the test–retest reliability of the WeChat questionnaire.

Methods:

A total of 120 women aged between 24 and 69 years were recruited from the outpatient clinic at Peking Union Medical College Hospital and randomly assigned to two groups. All participants completed the WeChat and paper questionnaires twice. Group A completed the paper questionnaire before the WeChat version; Group B completed the WeChat questionnaire before the paper version. Two weeks later, all participants completed the questionnaires in the opposite order. Then, the reliability and validity of the two versions were assessed using Pearson correlation coefficients, intraclass correlation coefficients, and Bland-Altman graphs.

Results:

No significant difference in completion time was found between the two versions of the Chinese PISQ-12 (P = 0.67). Half of the participants (60/120) preferred the WeChat questionnaire, 15% (18/120) preferred the paper form (P < 0.01), and 35% had no preference (42/120). The response time was positively correlated with age (P < 0.01) and negatively correlated with the degree of education (P < 0.01). A Pearson correlation coefficient of 0.92 and an intraclass correlation coefficient of 0.94 indicated strong consistency between the two versions. The WeChat form exhibited strong test–retest reliability (Pearson correlation coefficient, 0.86; intraclass correlation coefficient, 0.86). The Bland-Altman plots supported these results.

Conclusions:

The WeChat questionnaire was preferred over the paper version in a Chinese sample and had excellent consistency with the paper version and high test–retest reliability for collecting data on private topics.

End-user perspectives of two mHealth decision support tools: Electronic Community Case Management in Northern Malawi

BACKGROUND

The introduction of a paper-based Community Case Management (CCM) in Malawi has contributed to a reduction of child morbidity and mortality rates. In addition, the introduction of electronic Community Case Management (eCCM) (smartphones with built in CCM apps) may help to reduce the under-five mortality rates even further.

PURPOSE

It is not uncommon for Apps with a similar area of interest to develop different features to assist the end users. Such differences between Apps may have a significant role to play in its overall adoption and integration. The purpose of this research was to explore end users perspectives of two eCCM decision support tools developed and implemented by the Supporting LIFE project (SL eCCM App) and D-Tree International's (Mangologic eCCM App)in Northern Malawi.

METHODS

A mixed methods approach was applied, involving a survey of 109 users (106 Health Surveillance Assistants (HSAs), and 3 Integrated Management of Childhood Il6lnesses (IMCI) coordinators). This was followed up with semi-structured interviews with 34 respondents (31 HSAs, and 3 IMCI coordinators). Quantitative data was analyzed using SPSS version 20 where descriptive statistics and Chi-Squared tests were generated. Qualitative data were analyzed based on thematic analysis.

RESULTS

Participants reported that both Apps could assist the HSAs in the management of childhood illnesses. However, usability differed between the two apps where the Supporting LIFE eCCM App was found to be easier to use (61%) compared to the Mangologic eCCM App (4%). Both Apps were perceived to provide credible and accurate information.

CONCLUSION

It is essential that the quality of the data within Mobile Health (mHealth) Apps is high, however even Apps with excellent levels of data quality may not succeed if the overall usability of the App is low. Therefore it is essential that the Apps has high levels of data quality, usability and credibility. The results of this study will help inform mobile Health (mHealth) App designers in developing future eCCM Apps as well as researchers and policy makers when considering the adoption of mHealth solutions in the future in Malawi and other LMICs.

Digital Health Innovations, Tools, and Resources to Support Helping Babies Survive Programs

The Helping Babies Survive (HBS) initiative features a suite of evidence-based curricula and simulation-based training programs designed to provide health workers in low- and middle-income countries (LMICs) with the knowledge, skills, and competencies to prevent, recognize, and manage leading causes of newborn morbidity and mortality. Global scale-up of HBS initiatives has been rapid. As HBS initiatives rolled out across LMIC settings, numerous bottlenecks, gaps, and barriers to the effective, consistent dissemination and implementation of the programs, across both the pre- and in-service continuums, emerged. Within the first decade of expansive scale-up of HBS programs, mobile phone ownership and access to cellular networks have also concomitantly surged in LMICs. In this article, we describe a number of HBS digital health innovations and resources that have been developed from 2010 to 2020 to support education and training, data collection for monitoring and evaluation, clinical decision support, and quality improvement. Helping Babies Survive partners and stakeholders can potentially integrate the described digital tools with HBS dissemination and implementation efforts in a myriad of ways to support low-dose high-frequency skills practice, in-person refresher courses, continuing medical and nursing education, on-the-job training, or peer-to-peer learning, and strengthen data collection for key newborn care and quality improvement indicators and outcomes. Thoughtful integration of purpose-built digital health tools, innovations, and resources may assist HBS practitioners to more effectively disseminate and implement newborn care programs in LMICs, and facilitate progress toward the achievement of Sustainable Development Goal health goals, targets, and objectives.

An in-depth report of quality control on Kato-Katz and data entry in four clinical trials evaluating the efficacy of albendazole against soil-transmitted helminth infections

BACKGROUND

Efforts to control soil-transmitted helminth (STH) infections have intensified over the past decade. Field-survey data on STH prevalence, infection intensity and drug efficacy is necessary to guide the implementation of control programs and should be of the best possible quality.

METHODOLOGY

During four clinical trials designed to evaluate the efficacy of albendazole against STHs in Brazil, Ethiopia, Lao PDR and Tanzania, quality control (QC) was performed on the duplicate Kato-Katz thick smears and the data entry. We analyzed datasets following QC on both fecal egg counts (FECs) and data entry, and compared the prevalence of any STH infection and moderate-to-heavy intensity (MHI) infections and the drug efficacy against STH infections.

RESULTS

Across the four study sites, a total of 450 out of 4,830 (9.3%) Kato-Katz thick smears were re-examined. Discrepancies in FECs varied from ~3% (hookworms) to ~6.5% (Ascaris lumbricoides and Trichuris trichiura). The difference in STH prevalence and prevalence of MHI infections using the datasets with and without QC of the FECs did not exceed 0.3%, except for hookworm infections in Tanzania, where we noted a 2.2 percentage point increase in MHI infections (pre-QC: 1.6% vs. post-QC: 3.8%). There was a 100% agreement in the classification of drug efficacy of albendazole against STH between the two datasets. In total, 201 of the 28,980 (0.65%) data entries that were made to digitize the FECs were different between both data-entry clerks. Nevertheless, the overall prevalence of STH, the prevalence of MHI infections and the classification of drug efficacy remained largely unaffected.

CONCLUSION/SIGNIFICANCE

In these trials, where staff was informed that QC would take place, minimal changes in study outcomes were reported following QC on FECs or data entry. Nevertheless, imposing QC did reduce the number of errors. Therefore, application of QC together with proper training of the personnel and the availability of clear standard operating procedures is expected to support higher data quality.

Evaluation of a mobile health approach to improve the Early Warning System of influenza surveillance in Cameroon

Background

Rapid reporting of surveillance data is essential to better inform national prevention and control strategies.

Objectives

We compare the newly implemented smartphone‐based system to the former paper‐based and short message service (SMS) for collecting influenza epidemiological data in Cameroon.

Methods

Of the 13 sites which collect data from persons with influenza‐like illness (ILI), six sites send data through the EWS, while seven sites make use of the paper‐based system and SMS. We used four criteria for the comparison of the data collection tools: completeness, timeliness, conformity and cost.

Results

Regarding the different collection tools, data sent by the EWS were significantly more complete (97.6% vs 81.6% vs 44.8%), prompt (74.4% vs n/a vs 60.7%) and of better quality (93.7% vs 76.1% vs 84.0%) than data sent by the paper‐based system and SMS, respectively. The average cost of sending a datum by a sentinel site per week was higher for the forms (5.0 USD) than for the EWS (0.9 USD) and SMS (0.1 USD). The number of outpatient visits and subsequently all surveillance data decreased across the years 2017‐2019 together with the influenza positivity rate from 30.7% to 28.3%. Contrarily, the proportion of influenza‐associated ILI to outpatient load was highest in the year 2019 (0.37 per 100 persons vs 0.28 and 0.26 in the other 2 years).

Conclusion

All sentinel sites and even other disease surveillance systems are expected to use this tool in the near term future due to its satisfactory performance and cost.

Electronic data capture for large scale typhoid surveillance, household contact tracing, and health utilisation survey: Strategic Typhoid Alliance across Africa and Asia

Electronic data capture systems (EDCs) have the potential to achieve efficiency and quality in collection of multisite data. We quantify the volume, time, accuracy and costs of an EDC using large-scale census data from the STRATAA consortium, a comprehensive programme assessing population dynamics and epidemiology of typhoid fever in Malawi, Nepal and Bangladesh to inform vaccine and public health interventions. A census form was developed through a structured iterative process and implemented using Open Data Kit Collect running on Android-based tablets. Data were uploaded to Open Data Kit Aggregate, then auto-synced to MySQL-defined database nightly. Data were backed-up daily from three sites centrally, and auto-reported weekly. Pre-census materials’ costs were estimated. Demographics of 308,348 individuals from 80,851 households were recorded within an average of 14.7 weeks range (13-16) using 65 fieldworkers. Overall, 21.7 errors (95% confidence interval: 21.4, 22.0) per 10,000 data points were found: 13.0 (95% confidence interval: 12.6, 13.5) and 24.5 (95% confidence interval: 24.1, 24.9) errors on numeric and text fields respectively. These values meet standard quality threshold of 50 errors per 10,000 data points. The EDC’s total variable cost was estimated at US$13,791.82 per site. In conclusion, the EDC is robust, allowing for timely and high-volume accurate data collection, and could be adopted in similar epidemiological settings.

Costs and cost-effectiveness analyses of mCARE strategies for promoting care seeking of maternal and newborn health services in rural Bangladesh

Objective

We examined the incremental cost-effectiveness between two mHealth programs, implemented from 2011 to 2015 in rural Bangladesh: (1) Comprehensive mCARE package as an intervention group and (2) Basic mCARE package as a control group.

Methods

Both programs included a core package of census enumeration and pregnancy surveillance provided by an established cadre of digitally enabled community health workers (CHWs). In the comprehensive mCARE package, short message service (SMS) and home visit reminders were additionally sent to pregnant women (n = 610) and CHWs (n = 70) to promote the pregnant women’s care-seeking of essential maternal and newborn care services. Economic costs were assessed from a program perspective inclusive of development, start-up, and implementation phases. Effects were calculated as disability adjusted life years (DALYs) and the number of newborn deaths averted. For comparative purposes, we normalized our evaluation to estimate total costs and total newborn deaths averted per 1 million people in a community for both groups. Uncertainty was assessed using probabilistic sensitivity analyses with Monte Carlo simulation.

Results

The addition of SMS and home visit reminders based on a mobile phone-facilitated pregnancy surveillance system was highly cost effective at a cost per DALY averted of $31 (95% uncertainty range: $19–81). The comprehensive mCARE program had at least 88% probability of being highly cost-effective as compared to the basic mCARE program based on the threshold of Bangladesh’s GDP per capita.

Conclusion

mHealth strategies such as SMS and home visit reminders on a well-established pregnancy surveillance system may improve service utilization and program cost-effectiveness in low-resource settings.

Analysis of erroneous data entries in paper based and electronic data collection

OBJECTIVE

Electronic data collection (EDC) has become a suitable alternative to paper based data collection (PBDC) in biomedical research even in resource poor settings. During a survey in Nepal, data were collected using both systems and data entry errors compared between both methods. Collected data were checked for completeness, values outside of realistic ranges, internal logic and date variables for reasonable time frames. Variables were grouped into 5 categories and the number of discordant entries were compared between both systems, overall and per variable category.

RESULTS

Data from 52 variables collected from 358 participants were available. Discrepancies between both data sets were found in 12.6% of all entries (2352/18,616). Differences between data points were identified in 18.0% (643/3580) of continuous variables, 15.8% of time variables (113/716), 13.0% of date variables (140/1074), 12.0% of text variables (86/716), and 10.9% of categorical variables (1370/12,530). Overall 64% (1499/2352) of all discrepancies were due to data omissions, 76.6% (1148/1499) of missing entries were among categorical data. Omissions in PBDC (n = 1002) were twice as frequent as in EDC (n = 497, p < 0.001). Data omissions, specifically among categorical variables were identified as the greatest source of error. If designed accordingly, EDC can address this short fall effectively.

Patterns in On-time, Daily Submission of a Short Web-Based Personal Behavior Survey in a Longitudinal Women's Health Study

We evaluated compliance with submitting a short Web-based personal behavior survey daily during a 10-week study (n = 52 women/3419 diaries). Time-stamped forms revealed that 50% of diaries were submitted within 24 hours of the email prompt, and 19% were missing or submitted more than 3 days late. Late submissions may affect data quality.

Outbreak analytics: a developing data science for informing the response to emerging pathogens

Despite continued efforts to improve health systems worldwide, emerging pathogen epidemics remain a major public health concern. Effective response to such outbreaks relies on timely intervention, ideally informed by all available sources of data. The collection, visualization and analysis of outbreak data are becoming increasingly complex, owing to the diversity in types of data, questions and available methods to address them. Recent advances have led to the rise of outbreak analytics, an emerging data science focused on the technological and methodological aspects of the outbreak data pipeline, from collection to analysis, modelling and reporting to inform outbreak response. In this article, we assess the current state of the field. After laying out the context of outbreak response, we critically review the most common analytics components, their inter-dependencies, data requirements and the type of information they can provide to inform operations in real time. We discuss some challenges and opportunities and conclude on the potential role of outbreak analytics for improving our understanding of, and response to outbreaks of emerging pathogens. This article is part of the theme issue 'Modelling infectious disease outbreaks in humans, animals and plants: epidemic forecasting and control'. This theme issue is linked with the earlier issue 'Modelling infectious disease outbreaks in humans, animals and plants: approaches and important themes'.

Evaluation of Electronic and Paper-Pen Data Capturing Tools for Data Quality in a Public Health Survey in a Health and Demographic Surveillance Site, Ethiopia: Randomized Controlled Crossover Health Care Information Technology Evaluation

BACKGROUND

Periodic demographic health surveillance and surveys are the main sources of health information in developing countries. Conducting a survey requires extensive use of paper-pen and manual work and lengthy processes to generate the required information. Despite the rise of popularity in using electronic data collection systems to alleviate the problems, sufficient evidence is not available to support the use of electronic data capture (EDC) tools in interviewer-administered data collection processes.

OBJECTIVE

This study aimed to compare data quality parameters in the data collected using mobile electronic and standard paper-based data capture tools in one of the health and demographic surveillance sites in northwest Ethiopia.

METHODS

A randomized controlled crossover health care information technology evaluation was conducted from May 10, 2016, to June 3, 2016, in a demographic and surveillance site. A total of 12 interviewers, as 2 individuals (one of them with a tablet computer and the other with a paper-based questionnaire) in 6 groups were assigned in the 6 towns of the surveillance premises. Data collectors switched the data collection method based on computer-generated random order. Data were cleaned using a MySQL program and transferred to SPSS (IBM SPSS Statistics for Windows, Version 24.0) and R statistical software (R version 3.4.3, the R Foundation for Statistical Computing Platform) for analysis. Descriptive and mixed ordinal logistic analyses were employed. The qualitative interview audio record from the system users was transcribed, coded, categorized, and linked to the International Organization for Standardization 9241-part 10 dialogue principles for system usability. The usability of this open data kit-based system was assessed using quantitative System Usability Scale (SUS) and matching of qualitative data with the isometric dialogue principles.

RESULTS

From the submitted 1246 complete records of questionnaires in each tool, 41.89% (522/1246) of the paper and pen data capture (PPDC) and 30.89% (385/1246) of the EDC tool questionnaires had one or more types of data quality errors. The overall error rates were 1.67% and 0.60% for PPDC and EDC, respectively. The chances of more errors on the PPDC tool were multiplied by 1.015 for each additional question in the interview compared with EDC. The SUS score of the data collectors was 85.6. In the qualitative data response mapping, EDC had more positive suitability of task responses with few error tolerance characteristics.

CONCLUSIONS

EDC possessed significantly better data quality and efficiency compared with PPDC, explained with fewer errors, instant data submission, and easy handling. The EDC proved to be a usable data collection tool in the rural study setting. Implementation organization needs to consider consistent power source, decent internet connection, standby technical support, and security assurance for the mobile device users for planning full-fledged implementation and integration of the system in the surveillance site.

SMS-based smartphone application for disease surveillance has doubled completeness and timeliness in a limited-resource setting - evaluation of a 15-week pilot program in Central African Republic (CAR)

Background

It is a challenge in low-resource settings to ensure the availability of complete, timely disease surveillance information. Smartphone applications (apps) have the potential to enhance surveillance data transmission.

Methods

The Central African Republic (CAR) Ministry of Health and Médecins Sans Frontières (MSF) conducted a 15-week pilot project to test a disease surveillance app, Argus, for 20 conditions in 21 health centers in Mambéré Kadéi district (MK 2016). Results were compared to the usual paper-based surveillance in MK the year prior (MK 2015) and simultaneously in an adjacent health district, Nana-Mambére (NM 2016). Wilcoxon rank sum and Kaplan-Meier analyses compared report completeness and timeliness; the cost of the app, and users' perceptions of its usability were assessed.

Results

Two hundred seventy-one weekly reports sent by app identified 3403 cases and 63 deaths; 15 alerts identified 28 cases and 4 deaths. Median completeness (IQR) for MK 2016, 81% (81-86%), was significantly higher than in MK 2015 (31% (24-36%)), and NM 2016 (52% (48-57)) (p < 0.01). Median timeliness (IQR) for MK 2016, 50% (39-57%) was also higher than in MK 2015, 19% (19-24%), and NM 2016 29% (24-36%) (p < 0.01). Kaplan-Meier Survival Analysis showed a significant progressive reduction in the time taken to transmit reports over the 15-week period (p < 0.01). Users ranked the app's usability as greater than 4/5 on all dimensions. The total cost of the 15-week pilot project was US$40,575. It is estimated that to maintain the app in the 21 health facilities of MK will cost approximately US$18,800 in communication fees per year.

Conclusions

The app-based data transmission system more than doubled the completeness and timeliness of disease surveillance reports. This simple, low-cost intervention may permit the early detection of disease outbreaks in similar low-resource settings elsewhere.

Implementing electronic data capture at a well-established health and demographic surveillance site in rural northern Malawi

This article aims to assess multiple issues of resources, staffing, local opinion, data quality, cost, and security while transitioning to electronic data collection (EDC) at a long-running community research site in northern Malawi. Levels of missing and error fields, delay from data collection to availability, and average number of interviews per day were compared between EDC and paper in a complex, repeated annual household survey. Three focus groups with field and data staff with experience using both methods, and in-depth interviews with participants were carried out. Cost for each method were estimated and compared. Missing data was more common on paper questionnaires than on EDC, and a similar number were carried out per day. Fieldworkers generally preferred EDC, but data staff feared for their employment. Most respondents had no strong preference for a method. The cost of the paper system was estimated to be higher than using EDC. The existing infrastructure and technical expertise could be adapted to using EDC, but changes have an impact on data processing jobs as fewer, and better qualified staff are required. EDC is cost-effective, and, for a long-running site, may offer further savings, as devices can be used in multiple studies and perform several other functions. EDC is accepted by fieldworkers and respondents, has good levels of quality and timeliness, and security can be maintained. EDC is well-suited for use in a well-established research site using and developing existing infrastructure and expertise.

A New Influenza-Tracking Smartphone App (Flu-Report) Based on a Self-Administered Questionnaire: Cross-Sectional Study

BACKGROUND

Influenza infections can spread rapidly, and influenza outbreaks are a major public health concern worldwide. Early detection of signs of an influenza pandemic is important to prevent global outbreaks. Development of information and communications technologies for influenza surveillance, including participatory surveillance systems involving lay users, has recently increased. Many of these systems can estimate influenza activity faster than the conventional influenza surveillance systems. Unfortunately, few of these influenza-tracking systems are available in Japan.

OBJECTIVE

This study aimed to evaluate the flu-tracking ability of Flu-Report, a new influenza-tracking mobile phone app that uses a self-administered questionnaire for the early detection of influenza activity.

METHODS

Flu-Report was used to collect influenza-related information (ie, dates on which influenza infections were diagnosed) from November 2016 to March 2017. Participants were adult volunteers from throughout Japan, who also provided information about their cohabiting family members. The utility of Flu-Report was evaluated by comparison with the conventional influenza surveillance information and basic information from an existing large-scale influenza-tracking system (an automatic surveillance system based on electronic records of prescription drug purchases).

RESULTS

Information was obtained through Flu-Report for approximately 10,094 volunteers. In total, 2134 participants were aged <20 years, 6958 were aged 20-59 years, and 1002 were aged ≥60 years. Between November 2016 and March 2017, 347 participants reported they had influenza or an influenza-like illness in the 2016 season. Flu-Report-derived influenza infection time series data displayed a good correlation with basic information obtained from the existing influenza surveillance system (rho, ρ=.65, P=.001). However, the influenza morbidity ratio for our participants was approximately 25% of the mean influenza morbidity ratio for the Japanese population. The Flu-Report influenza morbidity ratio was 5.06% (108/2134) among those aged <20 years, 3.16% (220/6958) among those aged 20-59 years, and 0.59% (6/1002) among those aged ≥60 years. In contrast, influenza morbidity ratios for Japanese individuals aged <20 years, 20-59 years, and ≥60 years were recently estimated at 31.97% to 37.90%, 8.16% to 9.07%, and 2.71% to 4.39%, respectively.

CONCLUSIONS

Flu-Report supports easy access to near real-time information about influenza activity via the accumulation of self-administered questionnaires. However, Flu-Report users may be influenced by selection bias, which is a common issue associated with surveillance using information and communications technologies. Despite this, Flu-Report has the potential to provide basic data that could help detect influenza outbreaks.

A comparison of smartphone and paper data-collection tools in the Burden of Obstructive Lung Disease (BOLD) study in Gezira state, Sudan

Introduction

Data collection using paper-based questionnaires can be time consuming and return errors affect data accuracy, completeness, and information quality in health surveys. We compared smartphone and paper-based data collection systems in the Burden of Obstructive Lung Disease (BOLD) study in rural Sudan.

Methods

This exploratory pilot study was designed to run in parallel with the cross-sectional household survey. The Open Data Kit was used to programme questionnaires in Arabic into smartphones. We included 100 study participants (83% women; median age = 41.5 ± 16.4 years) from the BOLD study from 3 rural villages in East-Gezira and Kamleen localities of Gezira state, Sudan. Questionnaire data were collected using smartphone and paper-based technologies simultaneously. We used Kappa statistics and inter-rater class coefficient to test agreement between the two methods.

Results

Symptoms reported included cough (24%), phlegm (15%), wheezing (17%), and shortness of breath (18%). One in five were or had been cigarette smokers. The two data collection methods varied between perfect to slight agreement across the 204 variables evaluated (Kappa varied between 1.00 and 0.02 and inter-rater coefficient between 1.00 and -0.12). Errors were most commonly seen with paper questionnaires (83% of errors seen) vs smartphones (17% of errors seen) administered questionnaires with questions with complex skip-patterns being a major source of errors in paper questionnaires. Automated checks and validations in smartphone-administered questionnaires avoided skip-pattern related errors. Incomplete and inconsistent records were more likely seen on paper questionnaires.

Conclusion

Compared to paper-based data collection, smartphone technology worked well for data collection in the study, which was conducted in a challenging rural environment in Sudan. This approach provided timely, quality data with fewer errors and inconsistencies compared to paper-based data collection. We recommend this method for future BOLD studies and other population-based studies in similar settings.

Assessing stability and performance of a digitally enabled supply chain: Retrospective of a pilot in Uttar Pradesh, India

BACKGROUND

Immunization supply chains in low resource settings do not always reach children with necessary vaccines. Digital information systems can enable real time visibility of inventory and improve vaccine availability. In 2014, a digital, mobile/web-based information system was implemented in two districts of Uttar Pradesh, India. This retrospective investigates improvements and stabilization of supply chain performance following introduction of the digital information system.

METHODS

All data were collected via the digital information system between March 2014 and September 2015. Data included metadata and transaction logs providing information about users, facilities, and vaccines. Metrics evaluated include adoption (system access, timeliness and completeness), data quality (error rates), and performance (stock availability on immunization session days, replenishment response duration, rate of zero stock events). Stability was defined as the phase in which quality and performance metrics achieved equilibrium rates with minimal volatility. The analysis compared performance across different facilities and vaccines.

RESULTS

Adoption appeared sufficiently high from the onset to commence stability measures of data quality and supply chain performance. Data quality stabilized from month 3 onwards, and supply chain performance stabilized from month 13 onwards. For data quality, error rates reduced by two thirds post stabilization. Although vaccine availability remained high throughout the pilot, the three lowest-performing facilities improved from 91.05% pre-stability to 98.70% post-stability (p<0.01; t-test). Average replenishment duration (as a corrective response to stock-out events) decreased 52.3% from 4.93days to 2.35days (p<0.01; t-test). Diphtheria-tetanus-pertussis vaccine was significantly less likely to be stocked out than any other material.

CONCLUSION

The results suggest that given sufficient adoption, stability is sequentially achieved, beginning with data quality, and then performance. Identifying when a pilot stabilizes can enable more predictable, reliable cost estimates, and outcome forecasts in the scale-up phase.

Mobile electronic versus paper case report forms in clinical trials: a randomized controlled trial

BACKGROUND

Regulations, study design complexity and amounts of collected and shared data in clinical trials render efficient data handling procedures inevitable. Recent research suggests that electronic data capture can be key in this context but evidence is insufficient. This randomized controlled parallel group study tested the hypothesis that time efficiency is superior when electronic (eCRF) instead of paper case report forms (pCRF) are used for data collection. We additionally investigated predictors of time saving effects and data integrity.

METHODS

This study was conducted on top of a clinical weight loss trial performed at a clinical research facility over six months. All study nurses and patients participating in the clinical trial were eligible to participate and randomly allocated to enter cross-sectional data obtained during routine visits either through pCRF or eCRF. A balanced randomization list was generated before enrolment commenced. 90 and 30 records were gathered for the time that 27 patients and 2 study nurses required to report 2025 and 2037 field values, respectively. The primary hypothesis, that eCRF use is faster than pCRF use, was tested by a two-tailed t-test. Analysis of variance and covariance were used to evaluate predictors of entry performance. Data integrity was evaluated by descriptive statistics.

RESULTS

All randomized patients were included in the study (eCRF group n = 13, pCRF group n = 14). eCRF, as compared to pCRF, data collection was associated with significant time savings  across all conditions (8.29 ± 5.15 min vs. 10.54 ± 6.98 min, p = .047). This effect was not defined by participant type, i.e. patients or study nurses (F_(1,112) = .15, p = .699), CRF length (F_(2,112) = .49, p = .609) or patient age (Beta = .09, p = .534). Additional 5.16 ± 2.83 min per CRF were saved with eCRFs due to data transcription redundancy when patients answered questionnaires directly in eCRFs. Data integrity was superior in the eCRF condition (0 versus 3 data entry errors).

CONCLUSIONS

This is the first study to prove in direct comparison that using eCRFs instead of pCRFs increases time efficiency of data collection in clinical trials, irrespective of item quantity or patient age, and improves data quality.

TRIAL REGISTRATION

Clinical Trials NCT02649907 .

Electronic case report forms and electronic data capture within clinical trials and pharmacoepidemiology

AIMS

Researchers in clinical and pharmacoepidemiology fields have adopted information technology (IT) and electronic data capture, but these remain underused despite the benefits. This review discusses electronic case report forms and electronic data capture, specifically within pharmacoepidemiology and clinical research.

METHODS

The review used PubMed and the Institute of Electrical and Electronic Engineers library. Search terms used were agreed by the authors and documented. PubMed is medical and health based, whereas Institute of Electrical and Electronic Engineers is technology based. The review focuses on electronic case report forms and electronic data capture, but briefly considers other relevant topics; consent, ethics and security.

RESULTS

There were 1126 papers found using the search terms. Manual filtering and reviewing of abstracts further condensed this number to 136 relevant manuscripts. The papers were further categorized: 17 contained study data; 40 observational data; 27 anecdotal data; 47 covering methodology or design of systems; one case study; one literature review; two feasibility studies; and one cost analysis.

CONCLUSION

Electronic case report forms, electronic data capture and IT in general are viewed with enthusiasm and are seen as a cost-effective means of improving research efficiency, educating participants and improving trial recruitment, provided concerns about how data will be protected from misuse can be addressed. Clear operational guidelines and best practises are key for healthcare providers, and researchers adopting IT, and further work is needed on improving integration of new technologies with current systems. A robust method of evaluation for technical innovation is required.

Female respondent acceptance of computer-assisted personal interviewing (CAPI) for maternal, newborn and child health coverage surveys in rural Uganda

INTRODUCTION

High maternal and child mortality continues in low- and middle-income countries (LMIC). Measurement of maternal, newborn and child health (MNCH) coverage indicators often involves an expensive, complex, and lengthy household data collection process that is especially difficult in less-resourced settings. Computer-assisted personal interviewing (CAPI) has been proposed as a cost-effective and efficient alternative to traditional paper-and-pencil interviewing (PAPI). However, the literature on respondent-level acceptance of CAPI in LMIC has reported mixed outcomes. This is the first study to prospectively examine female respondent acceptance of CAPI and its influencing factors for MNCH data collection in rural Southwest Uganda.

METHODS

Eighteen women aged 15-49 years were randomly selected from 3 rural villages to participate. Each respondent was administered a Women's Questionnaire with half of the survey questions asked using PAPI techniques and the other half using CAPI. Following this PAPI/CAPI exposure, semi-structured focus group discussions (FGDs) assessed respondent attitudes towards PAPI versus CAPI. FGD data analysis involved an immersion/crystallization method (thematic narrative analysis).

RESULTS

The sixteen FGD respondents had a median age of 27 (interquartile range: 24.8, 32.3) years old. The majority (62.5%) had only primary level education. Most respondents (68.8%) owned or regularly used a mobile phone or computer. Few respondents (31.3%) had previously seen but not used a tablet computer. Overall, FGDs revealed CAPI acceptance and the factors influencing CAPI acceptability were 'familiarity', 'data confidentiality and security', 'data accuracy', and 'modernization and development'.

DISCUSSION

Female survey respondents in our rural Southwest Ugandan setting found CAPI to be acceptable. Global health planners and implementers considering CAPI for health coverage survey data collection should accommodate influencing factors during survey planning in order to maximize and facilitate acceptance and support by local stakeholders and community participants. Further research is needed to generate best practices for CAPI implementation and LMIC; higher quality, timely, streamlined and budget-friendly collection of MNCH indicators could help direct and improve programming to save lives of mothers and children.

Development and Implementation of Culturally Tailored Offline Mobile Health Surveys

Background

In low and middle income countries (LMICs), and other areas with low resources and unreliable access to the Internet, understanding the emerging best practices for the implementation of new mobile health (mHealth) technologies is needed for efficient and secure data management and for informing public health researchers. Innovations in mHealth technology can improve on previous methods, and dissemination of project development details and lessons learned during implementation are needed to provide lessons learned to stakeholders in both the United States and LMIC settings.

Objective

The aims of this paper are to share implementation strategies and lessons learned from the development and implementation stages of two survey research projects using offline mobile technology, and to inform and prepare public health researchers and practitioners to implement new mobile technologies in survey research projects in LMICs.

Methods

In 2015, two survey research projects were developed and piloted in Puerto Rico and pre-tested in Costa Rica to collect face-to-face data, get formative evaluation feedback, and to test the feasibility of an offline mobile data collection process. Fieldwork in each setting involved survey development, back translation with cultural tailoring, ethical review and approvals, data collector training, and piloting survey implementation on mobile tablets.

Results

Critical processes and workflows for survey research projects in low resource settings were identified and implemented. This included developing a secure mobile data platform tailored to each survey, establishing user accessibility, and training and eliciting feedback from data collectors and on-site LMIC project partners.

Conclusions

Formative and process evaluation strategies are necessary and useful for the development and implementation of survey research projects using emerging mHealth technologies in LMICs and other low resource settings. Lessons learned include: (1) plan institutional review board (IRB) approvals in multiple countries carefully to allow for development, implementation, and feedback, (2) in addition to testing the content of survey instruments, allow time and consideration for testing the use of novel mHealth technology (hardware and software), (3) incorporate training for and feedback from project staff, LMIC partner staff, and research participants, and (4) change methods accordingly, including content, as mHealth technology usage influences and is influenced by the content and structure of the survey instrument. Lessons learned from early phases of LMIC research projects using emerging mHealth technologies are critical for informing subsequent research methods and study designs.