Consensus Statement on Public Involvement and Engagement with Data Intensive Health Research

Evaluating participant experiences of Community Panels to scrutinise policy modelling for health inequalities: the SIPHER Consortium

Data-intensive research, including policy modelling, poses some distinctive challenges for efforts to mainstream public involvement into health research. There is a need for learning about how to design and deliver involvement for these types of research which are highly technical, and where researchers are at a distance from the people whose lives data depicts. This article describes our experiences involving members of the public in the SIPHER Consortium, a data-intensive policy modelling programme with researchers and policymakers working together over five years to try to address health inequalities. We focus on evaluating people's experiences as part of Community Panels for SIPHER. Key issues familiar from general public involvement efforts include practical details, careful facilitation of meetings, and payment for participants. We also describe some of the more particular learning around how to communicate technical research to non-academic audiences, in order to enable public scrutiny of research decisions. We conclude that public involvement in policy modelling can be meaningful and enjoyable, but that it needs to be carefully organised, and properly resourced.

Database quality assessment in research in paramedicine: a scoping review

BACKGROUND

Research in paramedicine faces challenges in developing research capacity, including access to high-quality data. A variety of unique factors in the paramedic work environment influence data quality. In other fields of healthcare, data quality assessment (DQA) frameworks provide common methods of quality assessment as well as standards of transparent reporting. No similar DQA frameworks exist for paramedicine, and practices related to DQA are sporadically reported. This scoping review aims to describe the range, extent, and nature of DQA practices within research in paramedicine.

METHODS

This review followed a registered and published protocol. In consultation with a professional librarian, a search strategy was developed and applied to MEDLINE (National Library of Medicine), EMBASE (Elsevier), Scopus (Elsevier), and CINAHL (EBSCO) to identify studies published from 2011 through 2021 that assess paramedic data quality as a stated goal. Studies that reported quantitative results of DQA using data that relate primarily to the paramedic practice environment were included. Protocols, commentaries, and similar study types were excluded. Title/abstract screening was conducted by two reviewers; full-text screening was conducted by two, with a third participating to resolve disagreements. Data were extracted using a piloted data-charting form.

RESULTS

Searching yielded 10,105 unique articles. After title and abstract screening, 199 remained for full-text review; 97 were included in the analysis. Included studies varied widely in many characteristics. Majorities were conducted in the United States (51%), assessed data containing between 100 and 9,999 records (61%), or assessed one of three topic areas: data, trauma, or out-of-hospital cardiac arrest (61%). All data-quality domains assessed could be grouped under 5 summary domains: completeness, linkage, accuracy, reliability, and representativeness.

CONCLUSIONS

There are few common standards in terms of variables, domains, methods, or quality thresholds for DQA in paramedic research. Terminology used to describe quality domains varied among included studies and frequently overlapped. The included studies showed no evidence of assessing some domains and emerging topics seen in other areas of healthcare. Research in paramedicine would benefit from a standardized framework for DQA that allows for local variation while establishing common methods, terminology, and reporting standards.

Participatory evaluation of the process of co-producing resources for the public on data science and artificial intelligence

BACKGROUND

The growth of data science and artificial intelligence offers novel healthcare applications and research possibilities. Patients should be able to make informed choices about using healthcare. Therefore, they must be provided with lay information about new technology. A team consisting of academic researchers, health professionals, and public contributors collaboratively co-designed and co-developed the new resource offering that information. In this paper, we evaluate this novel approach to co-production.

METHODS

We used participatory evaluation to understand the co-production process. This consisted of creative approaches and reflexivity over three stages. Firstly, everyone had an opportunity to participate in three online training sessions. The first one focused on the aims of evaluation, the second on photovoice (that included practical training on using photos as metaphors), and the third on being reflective (recognising one's biases and perspectives during analysis). During the second stage, using photovoice, everyone took photos that symbolised their experiences of being involved in the project. This included a session with a professional photographer. At the last stage, we met in person and, using data collected from photovoice, built the mandala as a representation of a joint experience of the project. This stage was supported by professional artists who summarised the mandala in the illustration.

RESULTS

The mandala is the artistic presentation of the findings from the evaluation. It is a shared journey between everyone involved. We divided it into six related layers. Starting from inside layers present the following experiences (1) public contributors had space to build confidence in a new topic, (2) relationships between individuals and within the project, (3) working remotely during the COVID-19 pandemic, (4) motivation that influenced people to become involved in this particular piece of work, (5) requirements that co-production needs to be inclusive and accessible to everyone, (6) expectations towards data science and artificial intelligence that researchers should follow to establish public support.

CONCLUSIONS

The participatory evaluation suggests that co-production around data science and artificial intelligence can be a meaningful process that is co-owned by everyone involved.

Eleven key measures for monitoring general practice clinical activity during COVID-19: A retrospective cohort study using 48 million adults' primary care records in England through OpenSAFELY

Background

The COVID-19 pandemic has had a significant impact on delivery of NHS care. We have developed the OpenSAFELY Service Restoration Observatory (SRO) to develop key measures of primary care activity and describe the trends in these measures throughout the COVID-19 pandemic.

Methods

With the approval of NHS England, we developed an open source software framework for data management and analysis to describe trends and variation in clinical activity across primary care electronic health record (EHR) data on 48 million adults.We developed SNOMED-CT codelists for key measures of primary care clinical activity such as blood pressure monitoring and asthma reviews, selected by an expert clinical advisory group and conducted a population cohort-based study to describe trends and variation in these measures January 2019-December 2021, and pragmatically classified their level of recovery one year into the pandemic using the percentage change in the median practice level rate.

Results

We produced 11 measures reflective of clinical activity in general practice. A substantial drop in activity was observed in all measures at the outset of the COVID-19 pandemic. By April 2021, the median rate had recovered to within 15% of the median rate in April 2019 in six measures. The remaining measures showed a sustained drop, ranging from a 18.5% reduction in medication reviews to a 42.0% reduction in blood pressure monitoring. Three measures continued to show a sustained drop by December 2021.

Conclusions

The COVID-19 pandemic was associated with a substantial change in primary care activity across the measures we developed, with recovery in most measures. We delivered an open source software framework to describe trends and variation in clinical activity across an unprecedented scale of primary care data. We will continue to expand the set of key measures to be routinely monitored using our publicly available NHS OpenSAFELY SRO dashboards with near real-time data.

Funding

This research used data assets made available as part of the Data and Connectivity National Core Study, led by Health Data Research UK in partnership with the Office for National Statistics and funded by UK Research and Innovation (grant ref MC_PC_20058).The OpenSAFELY Platform is supported by grants from the Wellcome Trust (222097/Z/20/Z); MRC (MR/V015757/1, MC_PC-20059, MR/W016729/1); NIHR (NIHR135559, COV-LT2-0073), and Health Data Research UK (HDRUK2021.000, 2021.0157).

Exploring how to improve the involvement of Polish and South Asian communities around big data research. A qualitative study using COM-B model

Introduction

Involving public contributors helps researchers to ensure that public views are taken into consideration when designing and planning research, so that it is person-centred and relevant to the public. This paper will consider public involvement in big data research. Inclusion of different communities is needed to ensure everyone's voice is heard. However, there remains limited evidence on how to improve the involvement of seldom-heard communities in big data research.

Objectives

This study aims to understand how South Asians and Polish communities in the UK can be encouraged to participate in public involvement initiatives in big data research.

Methods

Forty interviews were conducted with Polish (n=20) and South Asian (n=20) participants on Zoom. The participants were living in the United Kingdom and had not previously been involved as public contributors. Transcribed interviews were analysed using reflexive thematic analysis.

Results

We identified eight themes. The 'happy to reuse data' theme sets the scene by exploring our participants' views towards big data research and under what circumstances they thought that data could be used. The remaining themes were mapped under the capability-opportunity-motivation-behaviour (COM-B) model, as developed by Michie and colleagues. This allowed us to discuss multiple factors that could influence people's willingness to become public contributors.

Conclusions

Our study is the first to explore how to improve the involvement and engagement of seldom-heard communities in big data research using the COM-B model. The results have the potential to support researchers who want to identify what can influence members of the public to be involved. By using the COM-B model, it is possible to determine what measures could be implemented to better engage these communities.

A review of reviews exploring patient and public involvement in population health research and development of tools containing best practice guidance

INTRODUCTION

Patient and public involvement (PPI) is increasingly seen as something that is integral to research and of importance to research funders. There is general recognition that PPI is the right thing to do for both moral and practical reasons. The aim of this review of reviews is to examine how PPI can be done 'properly' by looking at the evidence that exists from published reviews and assessing it against the UK Standards for Public Involvement in Research, as well as examining the specific features of population health research that can make PPI more challenging.

METHODS

A review of reviews and development of best practice guidance was carried out following the 5-stage Framework Synthesis method.

RESULTS

In total 31 reviews were included. There is a lack of current research or clarity around Governance and Impact when findings are mapped against UK Standards for Public Involvement in Research. It was also clear that there is little knowledge around PPI with under-represented groups. There are gaps in knowledge about how to ensure key specific attributes of population health research are addressed for PPI team members - particularly around how to deal with complexity and the data-driven nature of the research. Four tools were produced for researchers and PPI members to further improve their PPI activity within population health research and health research more generally, including a framework of recommended actions to address PPI in population health research, and guidance on integrating PPI based on the UK Standards for Public Involvement in Research.

CONCLUSIONS

Facilitating PPI in population health research is challenging due to the nature of this type of research and there is far less evidence on how to do PPI well in this context. The tools can help researchers identify key aspects of PPI that can be integrated when designing PPI within projects. Findings also highlight specific areas where more research or discussion is needed.

OpenSAFELY NHS Service Restoration Observatory 2: changes in primary care clinical activity in England during the COVID-19 pandemic

BACKGROUND

The COVID-19 pandemic has disrupted healthcare activity across a broad range of clinical services. The NHS stopped non-urgent work in March 2020, later recommending services be restored to near-normal levels before winter where possible.

AIM

To describe changes in the volume and variation of coded clinical activity in general practice across six clinical areas: cardiovascular disease, diabetes, mental health, female and reproductive health, screening and related procedures, and processes related to medication.

DESIGN AND SETTING

With the approval of NHS England, a cohort study was conducted of 23.8 million patient records in general practice, in situ using OpenSAFELY.

METHOD

Common primary care activities were analysed using Clinical Terms Version 3 codes and keyword searches from January 2019 to December 2020, presenting median and deciles of code usage across practices per month.

RESULTS

Substantial and widespread changes in clinical activity in primary care were identified since the onset of the COVID-19 pandemic, with generally good recovery by December 2020. A few exceptions showed poor recovery and warrant further investigation, such as mental health (for example, for 'Depression interim review' the median occurrences across practices in December 2020 was down by 41.6% compared with December 2019).

CONCLUSION

Granular NHS general practice data at population-scale can be used to monitor disruptions to healthcare services and guide the development of mitigation strategies. The authors are now developing real-time monitoring dashboards for the key measures identified in this study, as well as further studies using primary care data to monitor and mitigate the indirect health impacts of COVID-19 on the NHS.

Understanding how to build a social licence for using novel linked datasets for planning and research in Kent, Surrey and Sussex: results of deliberative focus groups

Introduction

Digital programmes in the newly created NHS integrated care boards (ICBs) in the United Kingdom mean that curation and linkage of anonymised patient data is underway in many areas for the first time. In Kent, Surrey and Sussex (KSS), in Southeast England, public health teams want to use these datasets to answer strategic population health questions, but public expectations around use of patient data are unknown.

Objectives

We aimed to engage with citizens of KSS to gather their views and expectations of data linkage and re-use, through deliberative discussions.

Methods

We held five 3-hour deliberative focus groups with 79 citizens of KSS, presenting information about potential uses of data, safeguards, and mechanisms for public involvement in governance and decision making about datasets. After each presentation, participants discussed their views in facilitated small groups which were recorded, transcribed and analysed thematically.

Results

The focus groups generated 15 themes representing participants' views on the benefits, risks and values for safeguarding linked data. Participants largely supported use of patient data to improve health service efficiency and resource management, preventative services and out of hospital care, joined-up services and information flows. Most participants expressed concerns about data accuracy, breaches and hacking, and worried about commercial use of data. They suggested that transparency of data usage through audit trails and clear information about accountability, ensuring data re-use does not perpetuate stigma and discrimination, ongoing, inclusive and valued involvement of the public in dataset decision-making, and a commitment to building trust, would meet their expectations for responsible data use.

Conclusions

Participants were largely favourable about the proposed uses of patient linked datasets but expected a commitment to transparency and public involvement. Findings were mapped to previous tenets of social license and can be used to inform ICB digital programme teams on how to proceed with use of linked datasets in a trustworthy and socially acceptable way.

Setting up a Governance Framework for Secondary Use of Routine Health Data in Nursing Homes: Development Study Using Qualitative Interviews

BACKGROUND

In the nursing home sector, reusing routinely recorded data from electronic health records (EHRs) for knowledge development and quality improvement is still in its infancy. Trust in appropriate and responsible reuse is crucial for patients and nursing homes deciding whether to share EHR data for these purposes. A data governance framework determines who may access the data, under what conditions, and for what purposes. This can help obtain that trust. Although increasing attention is being paid to data governance in the health care sector, little guidance is available on development and implementation of a data governance framework in practice.

OBJECTIVE

This study aims to describe the development process of a governance framework for the "Registry Learning from Data in Nursing Homes," a national registry for EHR data on care delivered by nursing home physicians (in Dutch: specialist ouderengeneeskunde) in Dutch nursing homes-to allow data reusage for research and quality improvement of care.

METHODS

Relevant stakeholders representing practices, policies, and research in the nursing home sector were identified. Semistructured interviews were conducted with 20 people from 14 stakeholder organizations. The main aim of the interviews was to explore stakeholders' perspectives regarding the Registry's aim, data access criteria, and governing bodies' tasks and composition. Interview topics and analyses were guided by 8 principles regarding governance for reusing health data, as described in the literature. Interview results, together with legal advice and consensus discussions by the Registry's consortium partners, were used to shape the rules, regulations, and governing bodies of the governance framework.

RESULTS

Stakeholders valued the involvement of nursing home residents and their representatives, nursing home physicians, nursing homes' boards of directors, and scientists and saw this as a prerequisite for a trustworthy data governance framework. For the Registry, involvement of these groups can be achieved through a procedure in which residents can provide their consent or objection to the reuse of the data, transparency about the decisions made, and providing them a position in a governing body. In addition, a data request approval procedure based on predefined assessment criteria indicates that data reuse by third parties aligns with the aims of the Registry, benefits the nursing home sector, and protects the privacy of data subjects.

CONCLUSIONS

The stakeholders' views, expertise, and knowledge of other frameworks and relevant legislation serve to inform the application of governance principles to the contexts of both the nursing home sector and the Netherlands. Many different stakeholders were involved in the development of the Registry Learning from Data in Nursing Homes' governance framework and will continue to be involved. Engagement of the full range of stakeholders in an early stage of governance framework development is important to generate trust in appropriate and responsible data reuse.

CODE-EHR best practice framework for the use of structured electronic healthcare records in clinical research

Big data is central to new developments in global clinical science aiming to improve the lives of patients. Technological advances have led to the routine use of structured electronic healthcare records with the potential to address key gaps in clinical evidence. The covid-19 pandemic has demonstrated the potential of big data and related analytics, but also important pitfalls. Verification, validation, and data privacy, as well as the social mandate to undertake research are key challenges. The European Society of Cardiology and the BigData@Heart consortium have brought together a range of international stakeholders, including patient representatives, clinicians, scientists, regulators, journal editors and industry. We propose the CODE-EHR Minimum Standards Framework as a means to improve the design of studies, enhance transparency and develop a roadmap towards more robust and effective utilisation of healthcare data for research purposes.

Standardised data on initiatives-STARDIT: Beta version

BACKGROUND AND OBJECTIVE

There is currently no standardised way to share information across disciplines about initiatives, including fields such as health, environment, basic science, manufacturing, media and international development. All problems, including complex global problems such as air pollution and pandemics require reliable data sharing between disciplines in order to respond effectively. Current reporting methods also lack information about the ways in which different people and organisations are involved in initiatives, making it difficult to collate and appraise data about the most effective ways to involve different people. The objective of STARDIT (Standardised Data on Initiatives) is to address current limitations and inconsistencies in sharing data about initiatives. The STARDIT system features standardised data reporting about initiatives, including who has been involved, what tasks they did, and any impacts observed. STARDIT was created to help everyone in the world find and understand information about collective human actions, which are referred to as 'initiatives'. STARDIT enables multiple categories of data to be reported in a standardised way across disciplines, facilitating appraisal of initiatives and aiding synthesis of evidence for the most effective ways for people to be involved in initiatives. This article outlines progress to date on STARDIT; current usage; information about submitting reports; planned next steps and how anyone can become involved.

METHOD

STARDIT development is guided by participatory action research paradigms, and has been co-created with people from multiple disciplines and countries. Co-authors include cancer patients, people affected by rare diseases, health researchers, environmental researchers, economists, librarians and academic publishers. The co-authors also worked with Indigenous peoples from multiple countries and in partnership with an organisation working with Indigenous Australians.

RESULTS AND DISCUSSION

Over 100 people from multiple disciplines and countries have been involved in co-designing STARDIT since 2019. STARDIT is the first open access web-based data-sharing system which standardises the way that information about initiatives is reported across diverse fields and disciplines, including information about which tasks were done by which stakeholders. STARDIT is designed to work with existing data standards. STARDIT data will be released into the public domain (CC0) and integrated into Wikidata; it works across multiple languages and is both human and machine readable. Reports can be updated throughout the lifetime of an initiative, from planning to evaluation, allowing anyone to be involved in reporting impacts and outcomes. STARDIT is the first system that enables sharing of standardised data about initiatives across disciplines. A working Beta version was publicly released in February 2021 (ScienceforAll.World/STARDIT). Subsequently, STARDIT reports have been created for peer-reviewed research in multiple journals and multiple research projects, demonstrating the usability. In addition, organisations including Cochrane and Australian Genomics have created prospective reports outlining planned initiatives.

CONCLUSIONS

STARDIT can help create high-quality standardised information on initiatives trying to solve complex multidisciplinary global problems.

Governing Data and Artificial Intelligence for Health Care: Developing an International Understanding

Background

Although advanced analytical techniques falling under the umbrella heading of artificial intelligence (AI) may improve health care, the use of AI in health raises safety and ethical concerns. There are currently no internationally recognized governance mechanisms (policies, ethical standards, evaluation, and regulation) for developing and using AI technologies in health care. A lack of international consensus creates technical and social barriers to the use of health AI while potentially hampering market competition.

Objective

The aim of this study is to review current health data and AI governance mechanisms being developed or used by Global Digital Health Partnership (GDHP) member countries that commissioned this research, identify commonalities and gaps in approaches, identify examples of best practices, and understand the rationale for policies.

Methods

Data were collected through a scoping review of academic literature and a thematic analysis of policy documents published by selected GDHP member countries. The findings from this data collection and the literature were used to inform semistructured interviews with key senior policy makers from GDHP member countries exploring their countries’ experience of AI-driven technologies in health care and associated governance and inform a focus group with professionals working in international health and technology to discuss the themes and proposed policy recommendations. Policy recommendations were developed based on the aggregated research findings.

Results

As this is an empirical research paper, we primarily focused on reporting the results of the interviews and the focus group. Semistructured interviews (n=10) and a focus group (n=6) revealed 4 core areas for international collaborations: leadership and oversight, a whole systems approach covering the entire AI pipeline from data collection to model deployment and use, standards and regulatory processes, and engagement with stakeholders and the public. There was a broad range of maturity in health AI activity among the participants, with varying data infrastructure, application of standards across the AI life cycle, and strategic approaches to both development and deployment. A demand for further consistency at the international level and policies was identified to support a robust innovation pipeline. In total, 13 policy recommendations were developed to support GDHP member countries in overcoming core AI governance barriers and establishing common ground for international collaboration.

Conclusions

AI-driven technology research and development for health care outpaces the creation of supporting AI governance globally. International collaboration and coordination on AI governance for health care is needed to ensure coherent solutions and allow countries to support and benefit from each other’s work. International bodies and initiatives have a leading role to play in the international conversation, including the production of tools and sharing of practical approaches to the use of AI-driven technologies for health care.

Ethics as a Service: A Pragmatic Operationalisation of AI Ethics

As the range of potential uses for Artificial Intelligence (AI), in particular machine learning (ML), has increased, so has awareness of the associated ethical issues. This increased awareness has led to the realisation that existing legislation and regulation provides insufficient protection to individuals, groups, society, and the environment from AI harms. In response to this realisation, there has been a proliferation of principle-based ethics codes, guidelines and frameworks. However, it has become increasingly clear that a significant gap exists between the theory of AI ethics principles and the practical design of AI systems. In previous work, we analysed whether it is possible to close this gap between the ‘what’ and the ‘how’ of AI ethics through the use of tools and methods designed to help AI developers, engineers, and designers translate principles into practice. We concluded that this method of closure is currently ineffective as almost all existing translational tools and methods are either too flexible (and thus vulnerable to ethics washing) or too strict (unresponsive to context). This raised the question: if, even with technical guidance, AI ethics is challenging to embed in the process of algorithmic design, is the entire pro-ethical design endeavour rendered futile? And, if no, then how can AI ethics be made useful for AI practitioners? This is the question we seek to address here by exploring why principles and technical translational tools are still needed even if they are limited, and how these limitations can be potentially overcome by providing theoretical grounding of a concept that has been termed ‘Ethics as a Service.’

Public involvement and engagement in big data research: protocol for a scoping review and a systematic review of delivery and effectiveness of strategies for involvement and engagement

INTRODUCTION

Big data research has grown considerably over the last two decades. This presents new ethical challenges around consent, data storage and anonymisation. Big data research projects require public support to succeed and it has been argued that one way to achieve this is through public involvement and engagement. To better understand the role public involvement and engagement can play in big data research, we will review the current literature. This protocol describes the planned review methods.

METHODS AND ANALYSIS

Our review will be conducted in two stages. In the first stage, we will conduct a scoping review using Arksey and O'Malley methodology to comprehensively map current evidence on public involvement and engagement in big data research. Databases (CINAHL, Health Research Premium Collection, PubMed, Scopus, Web of Science) and grey literature will be searched for eligible papers. We provide a narrative description of the results based on a thematic analysis. In the second stage, out of papers found in the scoping review which discuss involvement and engagement strategies, we will conduct a systematic review following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, exploring the delivery and effectiveness of these strategies. We will conduct a qualitative synthesis. Relevant results from the quantitative studies will be extracted and placed under qualitative themes. Individual studies will be appraised through Mixed Methods Appraisal Tool (MMAT), we will then assess the overall confidence in each finding through Confidence in the Evidence from Reviews of Qualitative research (GRADE-CERQual). Results will be reported in a thematic and narrative way.

ETHICS AND DISSEMINATION

This protocol sets out how the review will be conducted to ensure rigour and transparency. Public advisors were involved in its development. Ethics approval is not required. Review findings will be presented at conferences and published in peer-reviewed journals.

The social licence for data-intensive health research: towards co-creation, public value and trust

Background

The rise of Big Data-driven health research challenges the assumed contribution of medical research to the public good, raising questions about whether the status of such research as a common good should be taken for granted, and how public trust can be preserved. Scandals arising out of sharing data during medical research have pointed out that going beyond the requirements of law may be necessary for sustaining trust in data-intensive health research. We propose building upon the use of a social licence for achieving such ethical governance.

Main text

We performed a narrative review of the social licence as presented in the biomedical literature. We used a systematic search and selection process, followed by a critical conceptual analysis. The systematic search resulted in nine publications. Our conceptual analysis aims to clarify how societal permission can be granted to health research projects which rely upon the reuse and/or linkage of health data. These activities may be morally demanding. For these types of activities, a moral legitimation, beyond the limits of law, may need to be sought in order to preserve trust. Our analysis indicates that a social licence encourages us to recognise a broad range of stakeholder interests and perspectives in data-intensive health research. This is especially true for patients contributing data. Incorporating such a practice paves the way towards an ethical governance, based upon trust. Public engagement that involves patients from the start is called for to strengthen this social licence.

Conclusions

There are several merits to using the concept of social licence as a guideline for ethical governance. Firstly, it fits the novel scale of data-related risks; secondly, it focuses attention on trustworthiness; and finally, it offers co-creation as a way forward. Greater trust can be achieved in the governance of data-intensive health research by highlighting strategic dialogue with both patients contributing the data, and the public in general. This should ultimately contribute to a more ethical practice of governance.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12910-021-00677-5.

Record linkage under suboptimal conditions for data-intensive evaluation of primary care in Rio de Janeiro, Brazil

Background

Linking Brazilian databases demands the development of algorithms and processes to deal with various challenges including the large size of the databases, the low number and poor quality of personal identifiers available to be compared (national security number not mandatory), and some characteristics of Brazilian names that make the linkage process prone to errors. This study aims to describe and evaluate the quality of the processes used to create an individual-linked database for data-intensive research on the impacts on health indicators of the expansion of primary care in Rio de Janeiro City, Brazil.

Methods

We created an individual-level dataset linking social benefits recipients, primary health care, hospital admission and mortality data. The databases were pre-processed, and we adopted a multiple approach strategy combining deterministic and probabilistic record linkage techniques, and an extensive clerical review of the potential matches. Relying on manual review as the gold standard, we estimated the false match (false-positive) proportion of each approach (deterministic, probabilistic, clerical review) and the missed match proportion (false-negative) of the clerical review approach. To assess the sensitivity (recall) to identifying social benefits recipients’ deaths, we used their vital status registered on the primary care database as the gold standard.

Results

In all linkage processes, the deterministic approach identified most of the matches. However, the proportion of matches identified in each approach varied. The false match proportion was around 1% or less in almost all approaches. The missed match proportion in the clerical review approach of all linkage processes were under 3%. We estimated a recall of 93.6% (95% CI 92.8–94.3) for the linkage between social benefits recipients and mortality data.

Conclusion

The adoption of a linkage strategy combining pre-processing routines, deterministic, and probabilistic strategies, as well as an extensive clerical review approach minimized linkage errors in the context of suboptimal data quality.

Closing the UK care home data gap - methodological challenges and solutions

UK care home residents are invisible in national datasets. The COVID-19 pandemic has exposed data failings that have hindered service development and research for years. Fundamental gaps, in terms of population and service demographics coupled with difficulties identifying the population in routine data are a significant limitation. These challenges are a key factor underpinning the failure to provide timely and responsive policy decisions to support care homes. In this commentary we propose changes that could address this data gap, priorities include: (1) Reliable identification of care home residents and their tenure; (2) Common identifiers to facilitate linkage between data sources from different sectors; (3) Individual-level, anonymised data inclusive of mortality irrespective of where death occurs; (4) Investment in capacity for large-scale, anonymised linked data analysis within social care working in partnership with academics; (5) Recognition of the need for collaborative working to use novel data sources, working to understand their meaning and ensure correct interpretation; (6) Better integration of information governance, enabling safe access for legitimate analyses from all relevant sectors; (7) A core national dataset for care homes developed in collaboration with key stakeholders to support integrated care delivery, service planning, commissioning, policy and research. Our suggestions are immediately actionable with political will and investment. We should seize this opportunity to capitalise on the spotlight the pandemic has thrown on the vulnerable populations living in care homes to invest in data-informed approaches to support care, evidence-based policy making and research.

Opening a "Can of Worms" to Explore the Public's Hopes and Fears About Health Care Data Sharing: Qualitative Study

BACKGROUND

Evidence suggests that health care data sharing may strengthen care coordination, improve quality and safety, and reduce costs. However, to achieve efficient and meaningful adoption of health care data-sharing initiatives, it is necessary to engage all stakeholders, from health care professionals to patients. Although previous work has assessed health care professionals' perceptions of data sharing, perspectives of the general public and particularly of seldom heard groups have yet to be fully assessed.

OBJECTIVE

This study aims to explore the views of the public, particularly their hopes and concerns, around health care data sharing.

METHODS

An original, immersive public engagement interactive experience was developed-The Can of Worms installation-in which participants were prompted to reflect about data sharing through listening to individual stories around health care data sharing. A multidisciplinary team with expertise in research, public involvement, and human-centered design developed this concept. The installation took place in three separate events between November 2018 and November 2019. A combination of convenience and snowball sampling was used in this study. Participants were asked to fill self-administered feedback cards and to describe their hopes and fears about the meaningful use of data in health care. The transcripts were compiled verbatim and systematically reviewed by four independent reviewers using the thematic analysis method to identify emerging themes.

RESULTS

Our approach exemplifies the potential of using interdisciplinary expertise in research, public involvement, and human-centered design to tell stories, collect perspectives, and spark conversations around complex topics in participatory digital medicine. A total of 352 qualitative feedback cards were collected, each reflecting participants' hopes and fears for health care data sharing. Thematic analyses identified six themes under hopes: enablement of personal access and ownership, increased interoperability and collaboration, generation of evidence for better and safer care, improved timeliness and efficiency, delivery of more personalized care, and equality. The five main fears identified included inadequate security and exploitation, data inaccuracy, distrust, discrimination and inequality, and less patient-centered care.

CONCLUSIONS

This study sheds new light on the main hopes and fears of the public regarding health care data sharing. Importantly, our results highlight novel concerns from the public, particularly in terms of the impact on health disparities, both at international and local levels, and on delivering patient-centered care. Incorporating the knowledge generated and focusing on co-designing solutions to tackle these concerns is critical to engage the public as active contributors and to fully leverage the potential of health care data use.

Examining the Use of Police and Ambulance Data in Suicide Research

Background: Police and paramedics are often the first to respond to individuals in suicide crisis and have an important role to play in facilitating optimal care pathways. Yet, little evidence exists to inform these responses. Data linkage provides one approach to examining this knowledge gap. Aim: We identified studies that examined suicide behaviors and linked to police or ambulance data. Method: A systematic search of PubMed and Scopus was undertaken to identify data linkage studies that: (1) examined suicide behaviors, and (2) included police or ambulance data. Studies were reviewed to identify: aims; suicide behaviors examined; how these were measured; how the cohort was defined; topic area; and what datasets were linked. Results: Eight studies met the inclusion criteria. Six studies included police data, and two studies included ambulance data. No study included both. Two topic areas were identified: (1) suicide-related contact with police or ambulance services; and (2) associations between suicidal behaviors and violence, victimization, and criminality. Limitations: Limitations to the review include the potential to have missed studies that investigated or reported on suicidality under the guise of mental health problems; complexities and nuances arising from the role of police data in coronial investigations; and limitations in the number of databases searched. Conclusion: Police and ambulance data represent a currently underutilized source of valuable information relevant to suicide crises, and further research should aim to address this gap.

Changes in public preferences for technologically enhanced surveillance following the COVID-19 pandemic: a discrete choice experiment

OBJECTIVES

As governments attempt to navigate a path out of COVID-19 restrictions, robust evidence is essential to inform requirements for public acceptance of technologically enhanced communicable disease surveillance systems. We examined the value of core surveillance system attributes to the Australian public, before and during the early stages of the current pandemic.

DESIGN

A discrete choice experiment was conducted in Australia with a representative group of respondents, before and after the WHO declared COVID-19 a Public Health Emergency of International Concern. We identified and investigated the relative importance of seven attributes associated with technologically enhanced disease surveillance: respect for personal autonomy; privacy/confidentiality; data certainty/confidence; data security; infectious disease mortality prevention; infectious disease morbidity prevention; and attribution of (causal) responsibility. Specifically, we explored how the onset of the COVID-19 outbreak influenced participant responses.

SETTING AND PARTICIPANTS

2008 Australians (general public) completed the experiment: 793 before COVID-19 outbreak onset (mean age 45.9 years, 50.2% male) and 1215 after onset (mean age 47.2 years, 49% male).

RESULTS

All seven attributes significantly influenced respondents' preferences for communicable disease surveillance systems. After onset, participants demonstrated greater preference for a surveillance system that could prevent a higher number of illnesses and deaths, and were less concerned about their personal autonomy. However, they also increased their preference for a system with high data security.

CONCLUSIONS

Public acceptance of technology-based communicable disease surveillance is situation dependent. During an epidemic, there is likely to be greater tolerance of technologically enhanced disease surveillance systems that result in restrictions on personal activity if such systems can prevent high morbidity and mortality. However, this acceptance of lower personal autonomy comes with an increased requirement to ensure data security. These findings merit further research as the pandemic unfolds and strategies are put in place that enable individuals and societies to live with SARS-CoV-2 endemicity.

Conditionally positive: a qualitative study of public perceptions about using health data for artificial intelligence research

OBJECTIVES

Given widespread interest in applying artificial intelligence (AI) to health data to improve patient care and health system efficiency, there is a need to understand the perspectives of the general public regarding the use of health data in AI research.

DESIGN

A qualitative study involving six focus groups with members of the public. Participants discussed their views about AI in general, then were asked to share their thoughts about three realistic health AI research scenarios. Data were analysed using qualitative description thematic analysis.

SETTINGS

Two cities in Ontario, Canada: Sudbury (400 km north of Toronto) and Mississauga (part of the Greater Toronto Area).

PARTICIPANTS

Forty-one purposively sampled members of the public (21M:20F, 25-65 years, median age 40).

RESULTS

Participants had low levels of prior knowledge of AI and mixed, mostly negative, perceptions of AI in general. Most endorsed using data for health AI research when there is strong potential for public benefit, providing that concerns about privacy, commercial motives and other risks were addressed. Inductive thematic analysis identified AI-specific hopes (eg, potential for faster and more accurate analyses, ability to use more data), fears (eg, loss of human touch, skill depreciation from over-reliance on machines) and conditions (eg, human verification of computer-aided decisions, transparency). There were mixed views about whether data subject consent is required for health AI research, with most participants wanting to know if, how and by whom their data were used. Though it was not an objective of the study, realistic health AI scenarios were found to have an educational effect.

CONCLUSIONS

Notwithstanding concerns and limited knowledge about AI in general, most members of the general public in six focus groups in Ontario, Canada perceived benefits from health AI and conditionally supported the use of health data for AI research.

The ethics of AI in health care: A mapping review

This article presents a mapping review of the literature concerning the ethics of artificial intelligence (AI) in health care. The goal of this review is to summarise current debates and identify open questions for future research. Five literature databases were searched to support the following research question: how can the primary ethical risks presented by AI-health be categorised, and what issues must policymakers, regulators and developers consider in order to be 'ethically mindful? A series of screening stages were carried out-for example, removing articles that focused on digital health in general (e.g. data sharing, data access, data privacy, surveillance/nudging, consent, ownership of health data, evidence of efficacy)-yielding a total of 156 papers that were included in the review. We find that ethical issues can be (a) epistemic, related to misguided, inconclusive or inscrutable evidence; (b) normative, related to unfair outcomes and transformative effectives; or (c) related to traceability. We further find that these ethical issues arise at six levels of abstraction: individual, interpersonal, group, institutional, and societal or sectoral. Finally, we outline a number of considerations for policymakers and regulators, mapping these to existing literature, and categorising each as epistemic, normative or traceability-related and at the relevant level of abstraction. Our goal is to inform policymakers, regulators and developers of what they must consider if they are to enable health and care systems to capitalise on the dual advantage of ethical AI; maximising the opportunities to cut costs, improve care, and improve the efficiency of health and care systems, whilst proactively avoiding the potential harms. We argue that if action is not swiftly taken in this regard, a new 'AI winter' could occur due to chilling effects related to a loss of public trust in the benefits of AI for health care.

Realist evaluation of public engagement and involvement in data-intensive health research

BACKGROUND

High quality public engagement and involvement (PEI) in data-intensive health research is seen as one way of ensuring that social legitimacy, i.e. a social license, is conferred through public acceptance of the need for research use of their data. This is a complex research area, and portfolios of involvement have been suggested, but not yet evaluated, to support the role of public contributors. The study aim was to evaluate if and how membership of a data-intensive research public forum can act as a mechanism for enhancing members' personal development. Our objective was to understand the circumstances and mechanisms that help to explain how, why and for whom involvement with a public forum enhanced those members' personal development.

METHOD

Qualitative data were collected from 15 current and previous members, via semi-structured interviews, notes from meetings, and consultations with and feedback from members. Data were critically compared, contrasted and reviewed until no new themes could be discerned and then condensed into context-mechanism-outcome (CMO) configurations. Realist evaluation was used to generate a theoretical and empirical appreciation of the contextual circumstances and mechanisms which help to explain the extent to which involvement with a public forum would enhance members' personal development and, if so, how, why, and for whom.

RESULTS

Three CMO configurations were identified. All of them showed that using the portfolio facilitated growth in forum members' personal development, but only where the members valued using the portfolio. This was particularly so for female members. Members valued the portfolio in one or more of three ways: as a tool to record and evidence activities, to facilitate reflective practice or as a guiding framework.

CONCLUSIONS

Data analysis and consideration of the three CMO configurations suggests a refined middle range theory that 'The use of a portfolio as a framework for learning in a public forum will facilitate members' personal development if they value its use as a framework for learning'. Further work is needed to confirm these findings both elsewhere in data-intensive health research and in other complex research areas using public forums for PEI.

PLAIN ENGLISH SUMMARY

Public engagement and involvement in health research is now well established and makes a valuable contribution to the research process. However, little is known about its impact on participants. This article investigates how involvement in a data-intensive health research public forum impacts on public forum members, rather than the research process. Personal involvement portfolios were used to support their involvement work and help evaluate if and how involvement in research activities enhanced members' personal development. Taking a realist evaluation approach, 'Context-Mechanism-Outcome' configurations were used to explore how membership of a public forum might enhance public forum members' personal development. The Context-Mechanism-Outcome configuration refers to an exploration of what influences the extent to which an intervention is successful or unsuccessful in producing positive outcomes and tries to identify the reasons why it is successful for some and unsuccessful for others. However, evidence from this realist evaluation recommends that engagement and involvement should always be underpinned by procedures which ensure that public contributors receive ongoing and tailored guidance and support throughout the process.