Mine, yours, ours? Sharing data on human genetic variation

Open science interventions to improve reproducibility and replicability of research: a scoping review

Various open science practices have been proposed to improve the reproducibility and replicability of scientific research, but not for all practices, there may be evidence they are indeed effective. Therefore, we conducted a scoping review of the literature on interventions to improve reproducibility. We systematically searched Medline, Embase, Web of Science, PsycINFO, Scopus and Eric, on 18 August 2023. Any study empirically evaluating the effectiveness of interventions aimed at improving the reproducibility or replicability of scientific methods and findings was included. We summarized the retrieved evidence narratively and in evidence gap maps. Of the 105 distinct studies we included, 15 directly measured the effect of an intervention on reproducibility or replicability, while the remainder addressed a proxy outcome that might be expected to increase reproducibility or replicability, such as data sharing, methods transparency or pre-registration. Thirty studies were non-comparative and 27 were comparative but cross-sectional observational designs, precluding any causal inference. Despite studies investigating a range of interventions and addressing various outcomes, our findings indicate that in general the evidence base for which various interventions to improve reproducibility of research remains remarkably limited in many respects.

Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data

OBJECTIVES

To synthesise research investigating data and code sharing in medicine and health to establish an accurate representation of the prevalence of sharing, how this frequency has changed over time, and what factors influence availability.

DESIGN

Systematic review with meta-analysis of individual participant data.

DATA SOURCES

Ovid Medline, Ovid Embase, and the preprint servers medRxiv, bioRxiv, and MetaArXiv were searched from inception to 1 July 2021. Forward citation searches were also performed on 30 August 2022.

REVIEW METHODS

Meta-research studies that investigated data or code sharing across a sample of scientific articles presenting original medical and health research were identified. Two authors screened records, assessed the risk of bias, and extracted summary data from study reports when individual participant data could not be retrieved. Key outcomes of interest were the prevalence of statements that declared that data or code were publicly or privately available (declared availability) and the success rates of retrieving these products (actual availability). The associations between data and code availability and several factors (eg, journal policy, type of data, trial design, and human participants) were also examined. A two stage approach to meta-analysis of individual participant data was performed, with proportions and risk ratios pooled with the Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis.

RESULTS

The review included 105 meta-research studies examining 2 121 580 articles across 31 specialties. Eligible studies examined a median of 195 primary articles (interquartile range 113-475), with a median publication year of 2015 (interquartile range 2012-2018). Only eight studies (8%) were classified as having a low risk of bias. Meta-analyses showed a prevalence of declared and actual public data availability of 8% (95% confidence interval 5% to 11%) and 2% (1% to 3%), respectively, between 2016 and 2021. For public code sharing, both the prevalence of declared and actual availability were estimated to be <0.5% since 2016. Meta-regressions indicated that only declared public data sharing prevalence estimates have increased over time. Compliance with mandatory data sharing policies ranged from 0% to 100% across journals and varied by type of data. In contrast, success in privately obtaining data and code from authors historically ranged between 0% and 37% and 0% and 23%, respectively.

CONCLUSIONS

The review found that public code sharing was persistently low across medical research. Declarations of data sharing were also low, increasing over time, but did not always correspond to actual sharing of data. The effectiveness of mandatory data sharing policies varied substantially by journal and type of data, a finding that might be informative for policy makers when designing policies and allocating resources to audit compliance.

SYSTEMATIC REVIEW REGISTRATION

Open Science Framework doi:10.17605/OSF.IO/7SX8U.

Characterizing the research data management practices of NIH biomedical researchers indicates the need for better support at laboratory level

BACKGROUND

The study investigated the research data management (RDM) practices of biomedical researchers at the National Institutes of Health (NIH) representing various biomedical disciplines.

OBJECTIVES

This study aimed to analyse the state of biomedical researchers' RDM practices based on RDM practice levels (individual, laboratory, institution and external). The findings of the study are expected to provide directions to information professionals for effective RDM services.

METHODS

Semi-structured interviews with 11 researchers were conducted. The interviews were analysed by levels of RDM practices.

RESULTS

The findings revealed that biomedical researchers focus on storing and sharing data and that RDM is performed mainly at the individual level. There seems to be a lack of laboratory level RDM system that allows consistent RDM practices among researchers. External RDM practice is often challenged by not having one responsible for RDM.

DISCUSSION

Findings suggested a need for an agreed RDM system and customized support, particularly at the laboratory level. Also, institutional support can help researchers prepare for long term data preservation.

CONCLUSION

Our suggestions emphasize the importance of RDM training and support for long term data preservation, especially at the laboratory level.

Reported Individual Costs and Benefits of Sharing Open Data among Canadian Academic Faculty in Ecology and Evolution

Open data facilitate reproducibility and accelerate scientific discovery but are hindered by perceptions that researchers bear costs and gain few benefits from publicly sharing their data, with limited empirical evidence to the contrary. We surveyed 140 faculty members working in ecology and evolution across Canada's top 20 ranked universities and found that more researchers report benefits (47.9%) and neutral outcomes (43.6%) than costs (21.4%) from openly sharing data. The benefits were independent of career stage and gender, but men and early career researchers were more likely to report costs. We outline mechanisms proposed by the study participants to reduce the individual costs and increase the benefits of open data for faculty members.

When are researchers willing to share their data? - Impacts of values and uncertainty on open data in academia

BACKGROUND

E-science technologies have significantly increased the availability of data. Research grant providers such as the European Union increasingly require open access publishing of research results and data. However, despite its significance to research, the adoption rate of open data technology remains low across all disciplines, especially in Europe where research has primarily focused on technical solutions (such as Zenodo or the Open Science Framework) or considered only parts of the issue.

METHODS AND FINDINGS

In this study, we emphasized the non-technical factors perceived value and uncertainty factors in the context of academia, which impact researchers' acceptance of open data-the idea that researchers should not only publish their findings in the form of articles or reports, but also share the corresponding raw data sets. We present the results of a broad quantitative analysis including N = 995 researchers from 13 large to medium-sized universities in Germany. In order to test 11 hypotheses regarding researchers' intentions to share their data, as well as detect any hierarchical or disciplinary differences, we employed a structured equation model (SEM) following the partial least squares (PLS) modeling approach.

CONCLUSIONS

Grounded in the value-based theory, this article proclaims that most individuals in academia embrace open data when the perceived advantages outweigh the disadvantages. Furthermore, uncertainty factors impact the perceived value (consisting of the perceived advantages and disadvantages) of sharing research data. We found that researchers' assumptions about effort required during the data preparation process were diminished by awareness of e-science technologies (such as Zenodo or the Open Science Framework), which also increased their tendency to perceive personal benefits via data exchange. Uncertainty factors seem to influence the intention to share data. Effects differ between disciplines and hierarchical levels.

The role of data-reuse experience in biological scientists’ data sharing: an empirical analysis

Purpose

The purpose of this paper is to investigate how scientists’ prior data-reuse experience affects their data-sharing intention by updating diverse attitudinal, control and normative beliefs about data sharing.

Design/methodology/approach

This paper used a survey method and the research model was evaluated by applying structural equation modelling to 476 survey responses from biological scientists in the USA.

Findings

The results show that prior data-reuse experience significantly increases the perceived community and career benefits and subjective norms of data sharing and significantly decreases the perceived risk and effort involved in data sharing. The perceived community benefits and subjective norms of data sharing positively influence scientists’ data-sharing intention, whereas the perceived risk and effort negatively influence scientists’ data-sharing intention.

Research limitations/implications

Based on the theory of planned behaviour, the research model was developed by connecting scientists’ prior data-reuse experience and data-sharing intention mediated through diverse attitudinal, control and normative perceptions of data sharing.

Practical implications

This research suggests that to facilitate scientists’ data-sharing behaviours, data reuse needs to be encouraged. Data sharing and reuse are interconnected, so scientists’ data sharing can be better promoted by providing them with data-reuse experience.

Originality/value

This is one of the initial studies examining the relationship between data-reuse experience and data-sharing behaviour, and it considered the following mediating factors: perceived community benefit, career benefit, career risk, effort and subjective norm of data sharing. This research provides an advanced investigation of data-sharing behaviour in the relationship with data-reuse experience and suggests significant implications for fostering data-sharing behaviour.

The formation of research ethics for data sharing by biological scientists: an empirical analysis

Purpose

The purpose of this paper is to investigate how biological scientists form research ethics for data sharing, and what the major factors affecting biological scientists’ formation of research ethics for data sharing are.

Design/methodology/approach

A research model for data sharing was developed based on the consequential theorists’ perspective of ethics. An online survey of 577 participants was administered, and the proposed research model was validated with a structural equation modeling technique.

Findings

The results show that egoism factors (perceived reputation, perceived risk, perceived effort), utilitarianism factors (perceived community benefit and perceived reciprocity) and norm of practice factors (perceived pressure by funding agency, perceived pressure by journal and norm of data sharing) all contribute to the formation of research ethics for data sharing.

Research limitations/implications

This research employed the consequentialist perspective of ethics for its research model development, and the proposed research model nicely explained how egoism, utilitarianism and norm of practice factors influence biological scientists’ research ethics for data sharing, which eventually leads to their data sharing intentions.

Practical implications

This research provides important practical implications for examining scientists’ data sharing behaviors from the perspective of research ethics. This research suggests that scientists’ data sharing behaviors can be better facilitated by emphasizing their egoism, utilitarianism and normative factors involved in research ethics for data sharing.

Originality/value

The ethical perspectives in data sharing research has been under-studied; this research sheds light on biological scientists’ formation of research ethics for data sharing, which can be applied in promoting scientists’ data sharing behaviors across different disciplines.

Multicenter data banking in management of dizzy patients: first results from the DizzyNet registry project

PURPOSE

Comprehensive phenotypical data across countries is needed to understand the determinants, prognosis and consequences of vestibular disease. The registry is a data repository for the members of the European DizzyNet. We report results from a pilot study using data from Turkey and Germany.

METHODS

The pilot study included a convenience sample of patients aged 18 or above referred to Ege University Medical School Hospital, Dokuz Eylül University Hospital, Izmir, Turkey, and the German Center for German Center for Vertigo and Balance Disorders, University on Munich, Germany, with symptoms of vertigo or dizziness. Health-related quality of life was assessed with the EQ5-D and the Dizziness Handicap Inventory (DHI). To obtain comparable groups we matched data from the two countries for age, sex and diagnosis by propensity score.

RESULTS

We included 80 adult patients, 40 from each country (60% female, mean age 54.1, SD 12.4). Matching was successful. Vestibular migraine (34%) was the most frequent diagnosis, followed by benign paroxysmal positional vertigo (29%) and Menière's disease (12%). Clinical signs and symptoms were comparable in both countries. Patients from Turkey were more likely to report headaches (65 vs. 32%) and to show gait unsteadiness (51 vs. 5%). Patients from Germany reported significantly higher quality of life and lower values of the DHI score.

CONCLUSIONS

Sharing data facilitates research, enhances translation from basic science into clinical applications, and increases transparency. The DizzyNet registry is a first step to data sharing in vestibular research across Europe.

To share or not to share? Expected pros and cons of data sharing in radiological research

The aims of this paper are to illustrate the trend towards data sharing, i.e. the regulated availability of the original patient-level data obtained during a study, and to discuss the expected advantages (pros) and disadvantages (cons) of data sharing in radiological research. Expected pros include the potential for verification of original results with alternative or supplementary analyses (including estimation of reproducibility), advancement of knowledge by providing new results by testing new hypotheses (not explored by the original authors) on pre-existing databases, larger scale analyses based on individual-patient data, enhanced multidisciplinary cooperation, reduced publication of false studies, improved clinical practice, and reduced cost and time for clinical research. Expected cons are outlined as the risk that the original authors could not exploit the entire potential of the data they obtained, possible failures in patients' privacy protection, technical barriers such as the lack of standard formats, and possible data misinterpretation. Finally, open issues regarding data ownership, the role of individual patients, advocacy groups and funding institutions in decision making about sharing of data and images are discussed.

KEY POINTS

• Regulated availability of patient-level data of published clinical studies (data-sharing) is expected. • Expected benefits include verification/advancement of knowledge, reduced cost/time of research, clinical improvement. • Potential drawbacks include faults in patients' identity protection and data misinterpretation.

Has open data arrived at the British Medical Journal (BMJ)? An observational study

OBJECTIVE

To quantify data sharing trends and data sharing policy compliance at the British Medical Journal (BMJ) by analysing the rate of data sharing practices, and investigate attitudes and examine barriers towards data sharing.

DESIGN

Observational study.

SETTING

The BMJ research archive.

PARTICIPANTS

160 randomly sampled BMJ research articles from 2009 to 2015, excluding meta-analysis and systematic reviews.

MAIN OUTCOME MEASURES

Percentages of research articles that indicated the availability of their raw data sets in their data sharing statements, and those that easily made their data sets available on request.

RESULTS

3 articles contained the data in the article. 50 out of 157 (32%) remaining articles indicated the availability of their data sets. 12 used publicly available data and the remaining 38 were sent email requests to access their data sets. Only 1 publicly available data set could be accessed and only 6 out of 38 shared their data via email. So only 7/157 research articles shared their data sets, 4.5% (95% CI 1.8% to 9%). For 21 clinical trials bound by the BMJ data sharing policy, the per cent shared was 24% (8% to 47%).

CONCLUSIONS

Despite the BMJ's strong data sharing policy, sharing rates are low. Possible explanations for low data sharing rates could be: the wording of the BMJ data sharing policy, which leaves room for individual interpretation and possible loopholes; that our email requests ended up in researchers spam folders; and that researchers are not rewarded for sharing their data. It might be time for a more effective data sharing policy and better incentives for health and medical researchers to share their data.

Samples and data accessibility in research biobanks: an explorative survey

Biobanks, which contain human biological samples and/or data, provide a crucial contribution to the progress of biomedical research. However, the effective and efficient use of biobank resources depends on their accessibility. In fact, making bio-resources promptly accessible to everybody may increase the benefits for society. Furthermore, optimizing their use and ensuring their quality will promote scientific creativity and, in general, contribute to the progress of bio-medical research. Although this has become a rather common belief, several laboratories are still secretive and continue to withhold samples and data. In this study, we conducted a questionnaire-based survey in order to investigate sample and data accessibility in research biobanks operating all over the world. The survey involved a total of 46 biobanks. Most of them gave permission to access their samples (95.7%) and data (85.4%), but free and unconditioned accessibility seemed not to be common practice. The analysis of the guidelines regarding the accessibility to resources of the biobanks that responded to the survey highlights three issues: (i) the request for applicants to explain what they would like to do with the resources requested; (ii) the role of funding, public or private, in the establishment of fruitful collaborations between biobanks and research labs; (iii) the request of co-authorship in order to give access to their data. These results suggest that economic and academic aspects are involved in determining the extent of sample and data sharing stored in biobanks. As a second step of this study, we investigated the reasons behind the high diversity of requirements to access biobank resources. The analysis of informative answers suggested that the different modalities of resource accessibility seem to be largely influenced by both social context and legislation of the countries where the biobanks operate.

Public Data Archiving in Ecology and Evolution: How Well Are We Doing?

Policies that mandate public data archiving (PDA) successfully increase accessibility to data underlying scientific publications. However, is the data quality sufficient to allow reuse and reanalysis? We surveyed 100 datasets associated with nonmolecular studies in journals that commonly publish ecological and evolutionary research and have a strong PDA policy. Out of these datasets, 56% were incomplete, and 64% were archived in a way that partially or entirely prevented reuse. We suggest that cultural shifts facilitating clearer benefits to authors are necessary to achieve high-quality PDA and highlight key guidelines to help authors increase their data's reuse potential and compliance with journal data policies.

Biomedical Data Sharing and Reuse: Attitudes and Practices of Clinical and Scientific Research Staff

Background

Significant efforts are underway within the biomedical research community to encourage sharing and reuse of research data in order to enhance research reproducibility and enable scientific discovery. While some technological challenges do exist, many of the barriers to sharing and reuse are social in nature, arising from researchers’ concerns about and attitudes toward sharing their data. In addition, clinical and basic science researchers face their own unique sets of challenges to sharing data within their communities. This study investigates these differences in experiences with and perceptions about sharing data, as well as barriers to sharing among clinical and basic science researchers.

Methods

Clinical and basic science researchers in the Intramural Research Program at the National Institutes of Health were surveyed about their attitudes toward and experiences with sharing and reusing research data. Of 190 respondents to the survey, the 135 respondents who identified themselves as clinical or basic science researchers were included in this analysis. Odds ratio and Fisher’s exact tests were the primary methods to examine potential relationships between variables. Worst-case scenario sensitivity tests were conducted when necessary.

Results and Discussion

While most respondents considered data sharing and reuse important to their work, they generally rated their expertise as low. Sharing data directly with other researchers was common, but most respondents did not have experience with uploading data to a repository. A number of significant differences exist between the attitudes and practices of clinical and basic science researchers, including their motivations for sharing, their reasons for not sharing, and the amount of work required to prepare their data.

Conclusions

Even within the scope of biomedical research, addressing the unique concerns of diverse research communities is important to encouraging researchers to share and reuse data. Efforts at promoting data sharing and reuse should be aimed at solving not only technological problems, but also addressing researchers’ concerns about sharing their data. Given the varied practices of individual researchers and research communities, standardizing data practices like data citation and repository upload could make sharing and reuse easier.

When data sharing gets close to 100%: what human paleogenetics can teach the open science movement

This study analyzes data sharing regarding mitochondrial, Y chromosomal and autosomal polymorphisms in a total of 162 papers on ancient human DNA published between 1988 and 2013. The estimated sharing rate was not far from totality (97.6% ± 2.1%) and substantially higher than observed in other fields of genetic research (evolutionary, medical and forensic genetics). Both a questionnaire-based survey and the examination of Journals' editorial policies suggest that this high sharing rate cannot be simply explained by the need to comply with stakeholders requests. Most data were made available through body text, but the use of primary databases increased in coincidence with the introduction of complete mitochondrial and next-generation sequencing methods. Our study highlights three important aspects. First, our results imply that researchers' awareness of the importance of openness and transparency for scientific progress may complement stakeholders' policies in achieving very high sharing rates. Second, widespread data sharing does not necessarily coincide with a prevalent use of practices which maximize data findability, accessibility, useability and preservation. A detailed look at the different ways in which data are released can be very useful to detect failures to adopt the best sharing modalities and understand how to correct them. Third and finally, the case of human paleogenetics tells us that a widespread awareness of the importance of Open Science may be important to build reliable scientific practices even in the presence of complex experimental challenges.

What drives academic data sharing?

Despite widespread support from policy makers, funding agencies, and scientific journals, academic researchers rarely make their research data available to others. At the same time, data sharing in research is attributed a vast potential for scientific progress. It allows the reproducibility of study results and the reuse of old data for new research questions. Based on a systematic review of 98 scholarly papers and an empirical survey among 603 secondary data users, we develop a conceptual framework that explains the process of data sharing from the primary researcher’s point of view. We show that this process can be divided into six descriptive categories: Data donor, research organization, research community, norms, data infrastructure, and data recipients. Drawing from our findings, we discuss theoretical implications regarding knowledge creation and dissemination as well as research policy measures to foster academic collaboration. We conclude that research data cannot be regarded as knowledge commons, but research policies that better incentivise data sharing are needed to improve the quality of research results and foster scientific progress.

Troubleshooting public data archiving: suggestions to increase participation

An increasing number of publishers and funding agencies require public data archiving (PDA) in open-access databases. PDA has obvious group benefits for the scientific community, but many researchers are reluctant to share their data publicly because of real or perceived individual costs. Improving participation in PDA will require lowering costs and/or increasing benefits for primary data collectors. Small, simple changes can enhance existing measures to ensure that more scientific data are properly archived and made publicly available: (1) facilitate more flexible embargoes on archived data, (2) encourage communication between data generators and re-users, (3) disclose data re-use ethics, and (4) encourage increased recognition of publicly archived data.

Why share data? Lessons learned from the fMRIDC

Neuroimaging and the discipline of cognitive neuroscience have grown together in lock-step with each pushing the other toward an improved ability to explore and examine brain function and form. However successful neuroimaging and the examination of cognitive processes may seem today, the culture of data sharing in these fields remains underdeveloped. In this article, we discuss our own experience in the development of the fMRI Data Center (fMRIDC) - a large-scale effort to gather, curate, and openly share the complete data sets from published research articles of brain activation studies using fMRI. We outline the fMRIDC effort's beginnings, how it operated, note some of the sociological reactions we received, and provide several examples of prominent new studies performed using data drawn from the archive. Finally, we provide comment on what considerations are needed for successful neuroimaging databasing and data sharing as existing and emerging efforts take the next steps in archiving and disseminating the field's valuable and irreplaceable data.