Evaluating the Quality and Usability of Open Data for Public Health Research: A Systematic Review of Data Offerings on 3 Open Data Platforms

Open government data in health sector: a systematic literature review open government data in health sector

BACKGROUND

Open government data (OGD) in the health sector consolidates transparency, access to information, and collaboration between the government and different sectors of society. It is an essential instrument for health systems and researchers to generate initiatives, drive innovations, and qualify decision-making, whether in health emergencies or supporting the creation of more effective public policies. This review aimed to identify OGD initiatives in healthcare and their possible applications.

METHODS

A systematic search was conducted in the MEDLINE (PubMed), Web of Science, Scopus, and Scielo databases up to February 12, 2024. Eligibility criteria included original studies that addressed OGD initiatives in public health. Studies using open data to prepare other analyses, reviews, and opinion articles were excluded. The quality methodological of related studies was based on the JBI Critical Appraisal tool. Data extraction included the location of the application, description of the initiative/use of data, themes addressed, data quality, and respective evaluation tools.

RESULTS

Of the 3,176 records initially identified, six studies were included describing initiatives to make available datasets or portals that publish OGD in the healthcare area between 2020 and 2023. Most publications evaluated quality and usability requirements, with five related to the COVID-19 pandemics. There is significant heterogeneity in the data formats available, obtaining, on average, a 3-star rating in the Berners-Lee evaluation model. The availability of open data presents geographic differences, with regions being more developed than others in both the most comprehensive and the most regionalized studies.

CONCLUSIONS

Few studies using OGD in health were identified in the scientific literature. The heterogeneity of open data formats, the lack of standards, and the quality of the data point to limitations and the need for improvements to enable the use and reliable reuse of this data.

TRIAL REGISTRATION

PROSPERO CRD42024498446.

A Standard Framework for Evaluating Large Health Care Data and Related Resources

Since 2000, the availability and use of large health care data and related resources for conducting surveillance, research, and evaluations to guide clinical and public health decision-making has increased rapidly. These trends have been related to transformations in health care information technology and public as well as private-sector efforts for collecting, compiling, and supplying large volumes of data. This growing collection of robust and often timely data has enhanced the capability to increase the knowledge base guiding clinical and public health activities and also has increased the need for effective tools to assess the attributes of these resources and identify the types of scientific questions they are best suited to address. This

MMWR supplement presents a standard framework for evaluating large health care data and related resources, including constructs, criteria, and tools that investigators and evaluators can apply and adapt.

Civic Utilisation based on meta-ethnography willingness of government to open data Modeling of Influential factors

[Purpose/Significance]Government open data has been in China for 10 years, however, the actual use of it has been criticised. This study analyses citizens' willingness to use government open data from their perspectives and constructs a model, aiming to promote in-depth research on citizens' use of government open data. [Methodology/Process] Meta-ethnographic qualitative research method was adopted to synthesise and integrate the 25 original studies. [Results/Conclusions] The study finally constructed a model of factors influencing citizens' willingness to use government open data, and identified four dimensions: "citizen", "platform", "government" and "environment". The four dimensions of "citizen", "platform", "government" and "environment" were identified, of which "citizen" involves "intrinsic motivation", "behavioural attitude", "information attitude" and "information security". The four dimensions of "Citizen" include "Intrinsic Motivation", "Behavioural Attitude", "Information Literacy", and "Demographic Characteristics", while the four dimensions of "Platform" include "Data Quality", "Information Literacy", and "Demographic Characteristics". platform" involves the sub-dimensions of "data quality" and "platform quality", "government" involves the sub-dimensions of "organisational conditions" and "policies and regulations", and "government" involves the sub-dimensions of "organisational conditions" and "policies and regulations". government" involves two sub-dimensions, "organisational conditions" and "policies and regulations", and "environment" involves two sub-dimensions, "social environment" and "type of context", with each sub-dimension having a different impact on the use of open government data by citizens. The different factors under each sub-dimension have different degrees of influence on citizens' use of open government data. Finally, suggestions are made to enhance citizens' willingness to use open government data.

Health system use and outcomes of urgently triaged callers to a nurse-managed telephone service for provincial health information after initiation of supplemental virtual physician assessment: a descriptive study

BACKGROUND

British Columbia's 8-1-1 telephone service connects callers with nurses for health care advice. As of Nov. 16, 2020, callers advised by a registered nurse to obtain in-person medical care can be subsequently referred to virtual physicians. We sought to determine health system use and outcomes of 8-1-1 callers urgently triaged by a nurse and subsequently assessed by a virtual physician.

METHODS

We identified callers referred to a virtual physician between Nov. 16, 2020, and Apr. 30, 2021. After assessment, virtual physicians assigned callers to 1 of 5 triage dispositions (i.e., go to emergency department [ED] now, see primary care provider within 24 hours, schedule an appointment with a health care provider, try home treatment, other). We linked relevant administrative databases to ascertain subsequent health care use and outcomes.

RESULTS

We identified 5937 encounters with virtual physicians involving 5886 8-1-1 callers. Virtual physicians advised 1546 callers (26.0%) to go to the ED immediately, of whom 971 (62.8%) had 1 or more ED visits within 24 hours. Virtual physicians advised 556 (9.4%) callers to seek primary care within 24 hours, of whom 132 (23.7%) had primary care billings within 24 hours. Virtual physicians advised 1773 (29.9%) callers to schedule an appointment with a health care provider, of whom 812 (45.8%) had primary care billings within 7 days. Virtual physicians advised 1834 (30.9%) callers to try a home treatment, of whom 892 (48.6%) had no health system encounters over the next 7 days. Eight (0.1%) callers died within 7 days of assessment with a virtual physician, 5 of whom were advised to go to the ED immediately. Fifty-four (2.9%) callers with a "try home treatment" disposition were admitted to hospital within 7 days of a virtual physician assessment, and no callers who were advised home treatment died.

INTERPRETATION

This Canadian study evaluated health service use and outcomes arising from the addition of virtual physicians to a provincial health information telephone service. Our findings suggest that supplementation of this service with an assessment from a virtual physician safely reduces the overall proportion of callers advised to seek urgent in-person visits.

Geospatial Analysis and Mapping Strategies for Fine-Grained and Detailed COVID-19 Data with GIS

The unprecedented COVID-19 pandemic is showing dramatic impact across the world. Public health authorities attempt to fight against the virus while maintaining economic activity. In the face of the uncertainty derived from the virus, all the countries have adopted non-pharmaceutical interventions for limiting the mobility and maintaining social distancing. In order to support these interventions, some health authorities and governments have opted for sharing very fine-grained data related with the impact of the virus in their territories. Geographical science is playing a major role in terms of understanding how the virus spreads across regions. Location of cases allows identifying the spatial patterns traced by the virus. Understanding these patterns makes controlling the virus spread feasible, minimizes its impact in vulnerable regions, anticipates potential outbreaks, or elaborates predictive risk maps. The application of geospatial analysis to fine-grained data must be urgently adopted for optimal decision making in real and near-real time. However, some aspects related to process and map sensitive health data in emergency cases have not yet been sufficiently explored. Among them include concerns about how these datasets with sensitive information must be shown depending on aspects related to data aggregation, scaling, privacy issues, or the need to know in advance the particularities of the study area. In this paper, we introduce our experience in mapping fine-grained data related to the incidence of the COVID-19 during the first wave in the region of Galicia (NW Spain), and after that we discuss the mentioned aspects.

Quality of government health data in COVID-19: definition and testing of an open government health data quality evaluation framework

Purpose

Currently, countries worldwide are struggling with the virus COVID-19 and the severe outbreak it brings. To better benefit from open government health data in the fight against this pandemic, this study developed a framework for assessing open government health data at the dataset level, providing a tool to evaluate current open government health data's quality and usability COVID-19.

Design/methodology/approach

Based on the review of the existing quality evaluation methods of open government data, the evaluation metrics and their weights were determined by 15 experts in health through the Delphi method and analytic hierarchy process. The authors tested the framework's applicability using open government health data related to COVID-19 in the US, EU and China.

Findings

The results of the test capture the quality difference of the current open government health data. At present, the open government health data in the US, EU and China lacks the necessary metadata. Besides, the number, richness of content and timeliness of open datasets need to be improved.

Originality/value

Unlike the existing open government data quality measurement, this study proposes a more targeted open government data quality evaluation framework that measures open government health data quality on a range of data quality dimensions with a fine-grained measurement approach. This provides a tool for accurate assessment of public health data for correct decision-making and assessment during a pandemic.

Evaluation of open health data portals for COVID-19 from the perspective of the user experience

Purpose

The purpose of this paper is to construct an indicator framework for evaluating open health data portals from the perspective of user experience (UX) to reduce users’ learning costs, save their time and energy and strengthen the emotional connection with users, thereby encouraging them to actively use open health data.

Design/methodology/approach

This study uses card sorting, Delphi and analytic hierarchy process to determine the weights of indicators for evaluating open health data portals. Then, this study uses a coding method to score, evaluate and compare the selection of more than 120 open health data portals supported by organizations in more than 100 countries or regions that are in the World's top confirmed cases of COVID-19 as released by the World Health Organization.

Findings

At present, open health data portals have shortcomings with regard to UX. Different types of open health data portals vary significantly in the dimensions of technical experience and functional experience, but the differences in the dimensions of aesthetic experience, emotional experience and content experience are not significant.

Originality/value

The constructed open health data portal evaluation indicator framework introduces users' actual application needs and proposes optimization suggestions for the portal to meet the needs of users to quickly obtain, reliable and accurate health data.

Business Owners’ Feedback toward Adoption of Open Data: A Case Study in Kuwait

The world intention toward open data technology has increased in the past years, and governments started to explore open data technology in the public and private sectors and tried to check its advantages and disadvantages. However, in the Arab world and especially in Kuwait, there is no solid structured attention about the technology in both sectors. As a result, we tried in this paper to determine if business owners in Kuwait have enough knowledge of the open data (OD) concept and if they have the willingness to use it for enhancing their services. The purpose of this research is to measure the acceptance of OD technology in Kuwait and to gather business owners' opinions about the ability to adopt the OD concept. Making online and hardcopy survey was our method for gathering different reactions and points of view about this technology. We intended to focus on the private sector and we targeted people who own a business and wish to introduce better services for their customers. Overall, the results have shown clear features about open data technology in Kuwait and the substantial need of education and awareness of the importance of this technology. The results of this study may positively and directly affect the level of motivation for other existing studies.

Life in Data Sets: Locating and Accessing Data on the Health of Americans Across the Life Span

CONTEXT

The US government manages a large number of data sets, including federally funded data collection activities that examine infectious and chronic conditions, as well as risk and protective factors for adverse health outcomes. Although there currently is no mature, comprehensive metadata repository of existing data sets, US federal agencies are working to develop and make metadata repositories available that will improve discoverability. However, because these repositories are not yet operating at full capacity, researchers must rely on their own knowledge of the field to identify available data sets.

PROGRAM OR POLICY

We sought to identify and consolidate a practical and annotated listing of those data sets.

IMPLEMENTATION AND/OR DISSEMINATION

Creative use of data resources to address novel questions is an important research skill in a wide range of fields including public health. This report identifies, promotes, and encourages the use of a range of data sources for health, behavior, economic, and policy research efforts across the life span.

EVALUATION

We identified and organized 28 federal data sets by the age-group of primary focus; not all groups are mutually exclusive. These data sets collectively represent a rich source of information that can be used to conduct descriptive epidemiologic studies.

DISCUSSION

The data sets identified in this article are not intended to represent an exhaustive list of all available data sets. Rather, we present an introduction/overview of the current federal data collection landscape and some of its largest and most frequently utilized data sets.

Repository Approaches to Improving the Quality of Shared Data and Code

Sharing data and code for reuse has become increasingly important in scientific work over the past decade. However, in practice, shared data and code may be unusable, or published results obtained from them may be irreproducible. Data repository features and services contribute significantly to the quality, longevity, and reusability of datasets. This paper presents a combination of original and secondary data analysis studies focusing on computational reproducibility, data curation, and gamified design elements that can be employed to indicate and improve the quality of shared data and code. The findings of these studies are sorted into three approaches that can be valuable to data repositories, archives, and other research dissemination platforms.

Extending an open-source tool to measure data quality: case report on Observational Health Data Science and Informatics (OHDSI)

Introduction

As the health system seeks to leverage large-scale data to inform population outcomes, the informatics community is developing tools for analysing these data. To support data quality assessment within such a tool, we extended the open-source software Observational Health Data Sciences and Informatics (OHDSI) to incorporate new functions useful for population health.

Methods

We developed and tested methods to measure the completeness, timeliness and entropy of information. The new data quality methods were applied to over 100 million clinical messages received from emergency department information systems for use in public health syndromic surveillance systems.

Discussion

While completeness and entropy methods were implemented by the OHDSI community, timeliness was not adopted as its context did not fit with the existing OHDSI domains. The case report examines the process and reasons for acceptance and rejection of ideas proposed to an open-source community like OHDSI.

Open government data, uncertainty and coronavirus: An infodemiological case study

Governments around the world have made data on COVID-19 testing, case numbers, hospitalizations and deaths openly available, and a breadth of researchers, media sources and data scientists have curated and used these data to inform the public about the state of the coronavirus pandemic. However, it is unclear if all data being released convey anything useful beyond the reputational benefits of governments wishing to appear open and transparent. In this analysis we use Ontario, Canada as a case study to assess the value of publicly available SARS-CoV-2 positive case numbers. Using a combination of real data and simulations, we find that daily publicly available test results probably contain considerable error about individual risk (measured as proportion of tests that are positive, population based incidence and prevalence of active cases) and that short term variations are very unlikely to provide useful information for any plausible decision making on the part of individual citizens. Open government data can increase the transparency and accountability of government, however it is essential that all publication, use and re-use of these data highlight their weaknesses to ensure that the public is properly informed about the uncertainty associated with SARS-CoV-2 information.

Moving towards open government data 2.0 in U.S. health agencies: Engaging data users and promoting use

We trace the evolution of open government data (OGD) publication among U.S. health agencies to illustrate how OGD goals and benefits might be achieved. Our novel conceptual framework illustrates the implicit logic underlying OGD activities in the health domain and their anticipated impact on population health. We conducted semi-structured interviews with 50 diverse practitioners and policymakers from local, state, and federal agencies, and non-governmental organizations. Using a positive deviance approach, we identified innovative U.S. health agencies that were early OGD adopters. We analyzed transcripts using a grounded theory methodological approach to identify common themes. Results indicate that the OGD movement is marked by three major eras (pre-OGD early activities, Open Data 1.0, and Open Data 2.0), and U.S. health agencies are in different stages of evolution. Among innovative jurisdictions, OGD transitioned from an early focus on releasing large volumes of data to a more demand-driven approach to promote meaningful user engagement with data. Although engagement strategies could yield benefits, limited evidence exists on best practices for engaging diverse data users and many jurisdictions have not yet transitioned to this later phase. Our conceptual framework could be adapted for other domains to help visualize how successful OGD initiatives might unfold.

Open data and its peers: understanding promising harbingers from Nordic Europe

Purpose

The peer-to-peer perspective on open data is an interesting topic to research, taking into account that data-driven innovations and related startups are often developed independently by civic and private stakeholders in a highly collaborative manner and are tentatively beginning to directly compete with traditional e-government solutions, providing arguably better services to citizens and businesses. In this regard, the paper aims to further debate on the potential of such independent data-driven collaboration not only to transform the traditional mechanisms of public sector innovations but also provide more democratic ways to ensure greater transparency of government and its responsibility before the society.

Design/methodology/approach

The research is based on a cross-country case study, resorting to the content analysis of three demonstrative cases in the development of open data-driven projects, which specifically promote peer-to-peer communication between its stakeholders. In this regard, the case study itself relies heavily on the analysis of rich empirical data that the author collected during his field studies in the Northern European region in 2015–2017, particularly in Estonia, Finland and Sweden. The practical research itself consists of three major parts, which reflect peer-to-peer perspectives of correspondingly civic, public and private stakeholders through manifested examples of related independent projects in the area.

Findings

The paper's results demonstrate that the use of peer-to-peer mechanisms in advancing related public sector reforms allows to transform the traditional understanding of e-government phenomena in a conceptually new way. E-government or its last more political interpretation – from the perspective of its peers could be regarded not necessarily as a platform to provide digital public services but as a source of raw material for various third party projects in, respectively, civic, government and business peer-to-peer dimensions of such reforms. As a result, open data provides an interesting playground to change the very nature of public sector innovations in the area.

Research limitations/implications

The choice of countries for research was motivated by purposive and convenience sampling because all these countries are situated in one region, have both similarities and differences in historical, political and socioeconomic backgrounds and, therefore, provide an ideal playground to investigate open data as a context dependable phenomenon. In this regard, the unique political and socioeconomic contexts of these countries provide an interesting playground to debate on the potential of social democracy, egalitarian society and social equality, i.e. public values that are deeply embedded into the fabric of societies there, to benefit the open data movement in a fundamental manner.

Practical implications

This paper reports on unique practical approaches for peer-to-peer collaboration and cooperation in advancing open data-driven platforms among stakeholders. The results of the case studies in three Nordic countries, which are currently among global leaders in advancing the concept of open government, are presented in an intrinsically illustrative manner, which could help practitioners and policymakers to understand better the potential of such a peer-to-peer perspective on open data. In this regard, the models proposed, of citizen-to-citizen, business-to-business, government-to-government interactions, could be interesting to a wide audience of e-government stakeholders in many nations.

Social implications

The paper also enters into philosophical debates about societal implications of digital peer-to-peer data-driven communication among people. Recent efforts to digitize almost every part of social life, starting from popularization of solutions for distant work and ending to online access to various public services, incentivize individual members of civil society to communicate in an inherently peer-to-peer way. This fact will definitely increase the demand for related digital services. Social distancing in a digital context will allow to paradoxically emancipate technically savvy and entrepreneurial people in creating new services, including using open data, which could meet the demand.

Originality/value

The research is intrinsically of an empirical character because recent e-government reforms in the public sector in many countries, including in the open data area, provide rich practical knowledge to test the limits of new technologies to advance society in socioeconomic and, more importantly, political development. In this regard, this paper provides the first research in analyzing open data from a unique peer-to-peer perspective with an ultimate goal of the whole investigation to draw the attention of other e-government scholars and initiate debates on the collaborative nature of the phenomena to empower civil society and ensure transparency of government.

How “accessible” is open data?

Purpose

This paper aims to examine the nature and sufficiency of descriptive information included in open datasets and the nature of comments and questions users write in relation to specific datasets. Open datasets are provided to facilitate civic engagement and government transparency. However, making the data available does not guarantee usage. This paper examined the nature of context-related information provided together with the datasets and identified the challenges users encounter while using the resources.

Design/methodology/approach

The authors extracted descriptive text provided together with (often at the top of) datasets (N = 216) and the nature of questions and comments users post in relation to the dataset. They then segmented text descriptions and user comments into “idea units” and applied open-coding with constant comparison method. This allowed them to come up with thematic issues that descriptions focus on and the challenges users encounter.

Findings

Results of the analysis revealed that context-related descriptions are limited and normative. Users are expected to figure out how to use the data. Analysis of user comments/questions revealed four areas of challenge they encounter: organization and accessibility of the data, clarity and completeness, usefulness and accuracy and language (spelling and grammar). Data providers can do more to address these issues.

Research limitations/implications

The purpose of the study is to understand the nature of open data provision and suggest ways of making open data more accessible to “non expert users”. As such, it is not focused on generalizing about open data provision in various countries as such provision may be different based on jurisdiction.

Practical implications

The study provides insight about ways of organizing open dataset that the resource can be accessible by the general public. It also provides suggestions about how open data providers could consider users' perspectives including providing continuous support.

Originality/value

Research on open data often focuses on technological, policy and political perspectives. Arguably, this is the first study on analysis of context-related information in open-datasets. Datasets do not “speak for themselves” because they require context for analysis and interpretation. Understanding the nature of context-related information in open dataset is original idea.

Research collaboration and knowledge sharing in e-governance

Purpose

This paper aims to introduce a systematic computing and analytical procedure that is applied to the co-author network to identify the temporal evolution and growth of research collaborations in the area of e-governance. The empirical analysis of the temporal co-author network can trace the emerging authors and knowledge bursts over time.

Design/methodology/approach

The study applied social network theory to trace the author collaboration patterns in the domain of e-governance. Analysis of the co-author network using micro and macro parameters was done to trace the temporal evolution of the author collaborations.

Findings

E-governance is a multi-disciplinary research domain split over streams of management, politics, information technology and electronics. Hence, research collaborations play a significant role in its advancement. The knowledge sharing between individual authors, institutions and groups through research collaborations, resulting in extensive sharing of data, equipment and research methods, has boosted research activities and development in e-governance. In this paper, the authors systematically analyse the current scenario of research collaborations in the area of e-governance using co-author network to estimate its impact on the advancement of the field. The authors also analysed the temporal evolution of the co-author networks, which show remarkable growth of research collaborations in the domain of e-governance from the year 2000.

Research limitations/implications

The co-author network analysis is only a proxy measure for the analysis of research collaborations. The names of the authors and the university affiliations used in the article are as retrieved from the research repository of Scopus. The degree, citations and other parameters related with authors have scope only within the environment of the co-author network used in the analysis. The criteria used in the study is limited to the degree of research collaborations and the number of co-authored publications in the giant component of the co-author network.

Practical implications

Institutions, authors and governments can trace and select suitable topics and choose research groups of co-authors over the world for future research collaborations in e-governance. The knowledge about the emerging and most discussed topics gives an overview of the global research trends of e-governance.

Social implications

The study identified the evolution of creative collaborations in e-governance in the global perspective. The methodology introduced here is helpful to detect the proficient and productive author collaborations and the spectrum of related e-governance research topics associated with them. As the author collaborations can be mapped to the institutional and country-level collaborations, the information is helpful for researchers, institutions and governments to establish the best collaborations in e-governance research based on the author proficiency, collaboration patterns and research topics as per the requirements.

Originality/value

The paper introduces a novel research methodology using temporal analysis of co-author network to identify the evolution of research patterns and the associated research topics.

A beginner's guide to data stewardship and data sharing

STUDY DESIGN

A narrative review of principles, benefits and disadvantages, as well as methods of research data sharing.

OBJECTIVES

To assist prospective Spinal Cord authors and others with understanding and implementing data sharing, so that various benefits of such sharing can accrue to all spinal cord injury research stakeholders.

SETTING

International.

METHODS

The medical research and health care services literature was reviewed nonsystematically for relevant articles, and web sites were explored for information and services offered by various pertinent organizations.

RESULTS

Grant makers, professional organizations, research journals, publishers, and other entities in the research field increasingly stress the ethics as well as societal and practical benefits of data sharing, and require researchers to do so within a reasonable time after data collection ends. Sharing data, retrospectively, generally requires much time and resources, but when a data management plan is part of a research proposal from the start, costs are limited, and grant makers allow these costs to be part of a budget. There are many organizations that offer information on or even assist with preparing data for sharing and actual deposit in a data repository.

CONCLUSIONS

The requirement of data sharing is not likely to go away, and researchers interested in submitting their reports to Spinal Cord would do well to familiarize themselves with the myriad practical issues involved in preparing data for sharing.

Open data and public health

This article provides an overview of the intersection of open data and public health by first defining open government data, public health data, and other key concepts and relevant terminologies. There are differing perceptions on the urgency and importance of the openness of public health data. It has been established that disease outbreaks such as happened during the Ebola and Zika virus epidemics are indicative of the need for countries to develop a framework that will provide guidance for the management of public health data. Such a framework should ensure that data collected during public health emergencies are accessible to the appropriate authorities and in a form that can help with timely decision-making during such public health crises. In this article, we highlight available open data policies across many countries, including in the Americas. Our analysis shows that there are currently no articulated policy guidelines for the collection and management of public health data across many countries, especially in Latin America. We propose that any national data governance strategy must address potential benefits, possible risks, examples of data that could be shared, and the attributes of such data. Finally, we stress that the key concern in the Americas should be the development of regional frameworks for open data in public health that can be adopted or adapted by each country through appropriate national or subnational policies and strategies.