Survey on Scientific Shared Resource Rigor and Reproducibility

Improving light microscopy training routines with evidence-based education

The low reproducibility of scientific data published in articles has recently become a cause of concern in many scientific fields. Data involving light microscopy is no exception. The low awareness of researchers of the technologies they use in their research has been identified as one of the main causes of the problem. Potential solutions have hinted at the need to improve technological and methodological education within research. Despite the pivotal role of microscopy core facilities in the education of researchers being well documented, facility staff (FS) often learn their trade on the job, without receiving themselves any structured education about the technology they teach others to use. Additionally, despite endorsing an important role at the highest level of education, most FS never receive any training in pedagogy, the field of research on teaching and learning methods. In this article, we argue that the low level of awareness that researchers have of microscopy stems from a knowledge gap formed between them and microscopy FS during training routines. On the one hand, FS consider that their teaching task is to explain what is needed to produce reliable data. On the other, despite understanding what is being taught, researchers fail to learn the most challenging aspects of microscopy, those involving their judgement and reasoning. We suggest that the misunderstanding between FS and researchers is due to FS not being educated in pedagogy and thus often confusing understanding and learning. To bridge this knowledge gap and improve the quality of the microscopy education available to researchers, we propose a paradigm shift where training staff at technological core facilities be acknowledged as full-fledged teachers and offered structured education not only in the technology they teach but also in pedagogy. We then suggest that training routines at facilities be upgraded to follow the principles of the Constructive Alignment pedagogical method. We give an example of how this can be applied to existing microscopy training routines. We also describe a model to define where the responsibility of FS in training researchers begins and ends. This involves a major structural change where university staff involved in teaching research technologies themselves receive appropriate education. For this to be achieved, we advocate that funding agencies, universities, microscopy and core facility organisations mobilise resources of time and funding. Such changes may involve funding the creation and development of 'Train-the-trainer' type of courses and giving incentives for FS to upgrade their technological and pedagogical knowledge, for example by including them in career paths. We believe that this paradigm shift is necessary to improve the level of microscopy education and ultimately the reproducibility of published data.

Exploring pathways toward open-hardware ecosystems to safeguard genetic resources for biomedical research communities using aquatic model species

Development of reliable germplasm repositories is critical for preservation of genetic resources of aquatic species, which are widely utilized to support biomedical innovation by providing a foundational source for naturally occurring variation and development of new variants through genetic manipulations. A significant barrier in repository development is the lack of cryopreservation capability and reproducibility across the research community, posing great risks of losing advances developed from billions of dollars of research investment. The emergence of open scientific hardware has fueled a new movement across biomedical research communities. With the increasing accessibility of consumer-level fabrication technologies, such as three-dimensional printers, open hardware devices can be custom designed, and design files distributed to community members for enhancing rigor, reproducibility, and standardization. The overall goal of this review is to explore pathways to create open-hardware ecosystems among the communities using aquatic model resources for biomedical research. To gain feedback and insights from community members, an interactive workshop focusing on open-hardware applications in germplasm repository development was held at the 2022 Aquatic Models for Human Disease Conference, Woods Hole, Massachusetts. This work integrates conceptual strategies with practical insights derived from workshop interactions using examples of germplasm repository development. These insights can be generalized for establishment of open-hardware ecosystems for a broad biomedical research community. The specific objectives were to: (1) introduce an open-hardware ecosystem concept to support biomedical research; (2) explore pathways toward open-hardware ecosystems through four major areas, and (3) identify opportunities and future directions.

Tissue-Like 3D Standard and Protocols for Microscope Quality Management

This article outlines a global study conducted by the Association of Biomedical Resource Facilities (ABRF) Light Microscopy Research Group (LMRG). The results present a novel 3D tissue-like biologically relevant standard sample that is affordable and straightforward to prepare. Detailed sample preparation, instrument-specific image acquisition protocols and image analysis methods are presented and made available to the community. The standard consists of sub-resolution and large well characterized relative intensity fluorescence microspheres embedded in a 120 µm thick 3D gel with a refractive index of 1.365. The standard allows the evaluation of several properties as a function of depth. These include the following: 1) microscope resolution with automated analysis of the point-spread function (PSF), 2) automated signal-to-noise ratio analysis, 3) calibration and correction of fluorescence intensity loss, and 4) quantitative relative intensity. Results demonstrate expected refractive index mismatch dependent losses in intensity and resolution with depth, but the relative intensities of different objects at similar depths are maintained. This is a robust standard showing reproducible results across laboratories, microscope manufacturers and objective lens types (e.g., magnification, immersion medium). Thus, these tools will be valuable for the global community to benchmark fluorescence microscopes and will contribute to improved scientific rigor and reproducibility.

Tissue Engineering Challenges for Cultivated Meat to Meet the Real Demand of a Global Market

Cultivated meat (CM) technology has the potential to disrupt the food industry-indeed, it is already an inevitable reality. This new technology is an alternative to solve the environmental, health and ethical issues associated with the demand for meat products. The global market longs for biotechnological improvements for the CM production chain. CM, also known as cultured, cell-based, lab-grown, in vitro or clean meat, is obtained through cellular agriculture, which is based on applying tissue engineering principles. In practice, it is first necessary to choose the best cell source and type, and then to furnish the necessary nutrients, growth factors and signalling molecules via cultivation media. This procedure occurs in a controlled environment that provides the surfaces necessary for anchor-dependent cells and offers microcarriers and scaffolds that favour the three-dimensional (3D) organisation of multiple cell types. In this review, we discuss relevant information to CM production, including the cultivation process, cell sources, medium requirements, the main obstacles to CM production (consumer acceptance, scalability, safety and reproducibility), the technological aspects of 3D models (biomaterials, microcarriers and scaffolds) and assembly methods (cell layering, spinning and 3D bioprinting). We also provide an outlook on the global CM market. Our review brings a broad overview of the CM field, providing an update for everyone interested in the topic, which is especially important because CM is a multidisciplinary technology.

Banking on a new understanding: translational opportunities from veterinary biobanks

Current advances in geroscience are due in part to the discovery of biomarkers with high predictive ability in short-lived laboratory animals such as flies and mice. These model species, however, do not always adequately reflect human physiology and disease, highlighting the need for a more comprehensive and relevant model of human aging. Domestic dogs offer a solution to this obstacle, as they share many aspects not only of the physiological and pathological trajectories of their human counterpart, but also of their environment. Furthermore, they age at a considerably faster rate. Studying aging in the companion dog provides an opportunity to better understand the biological and environmental determinants of healthy lifespan in our pets, and to translate those findings to human aging. Biobanking, the systematic collection, processing, storage, and distribution of biological material and associated data has contributed to basic, clinical, and translational research by streamlining the management of high-quality biospecimens for biomarker discovery and validation. In this review, we discuss how veterinary biobanks can support research on aging, particularly when integrated into large-scale longitudinal studies. As an example of this concept, we introduce the Dog Aging Project Biobank.

Rigor, Reproducibility, and Transparency in Shared Research Resources: Follow-Up Survey and Recommendations for Improvements

Rigor, reproducibility, and transparency (RR&T) are essential components of all scientific pursuits. Shared research resources, also known as core facilities, are on the frontlines of ensuring robust RR&T practices. The Association of Biomolecular Resource Facilities Committee on Core Rigor and Reproducibility conducted a follow-up survey 4 years after the initial 2017 survey to determine if core facilities have seen a positive impact of new RR&T initiatives (including guidance from the National Institutes of Health, new scientific journal requirements on transparency and data provenance, and educational tools from professional organizations). While there were fewer participants in the most recent survey, the respondents' opinions on the role of core facilities and level of best practices adoption remained the same. Overall, the respondents agreed that procedures should be implemented by core facilities to ensure scientific RR&T. They also indicated that there is a strong correlation between institutions that emphasize RR&T and core customers using this expertise in grant applications and publications. The survey also assessed the impact of the COVID-19 pandemic on core operations and RR&T. The answers to these pandemic-related questions revealed that many of the strategies aimed at increasing efficiencies are also best practices related to RR&T, including the development of standard operating procedures, supply chain management, and cross training. Given the consistent and compelling awareness of the importance of RR&T expressed by core directors in 2017 and 2021 contrasted with the lack of apparent improvements over this time period, the authors recommend an adoption of RR&T statements by all core laboratories. Adhering to the RR&T guidelines will result in more efficient training, better compliance, and improved experimental approaches empowering cores to become "rigor champions."

Research data management for bioimaging: the 2021 NFDI4BIOIMAGE community survey

Background:  Knowing the needs of the bioimaging community with respect to research data management (RDM) is essential for identifying measures that enable adoption of the FAIR (findable, accessible, interoperable, reusable) principles for microscopy and bioimage analysis data across disciplines. As an initiative within Germany's National Research Data Infrastructure, we conducted this community survey in summer 2021 to assess the state of the art of bioimaging RDM and the community needs. Methods: An online survey was conducted with a mixed question-type design. We created a questionnaire tailored to relevant topics of the bioimaging community, including specific questions on bioimaging methods and bioimage analysis, as well as more general questions on RDM principles and tools. 203 survey entries were included in the analysis covering the perspectives from various life and biomedical science disciplines and from participants at different career levels. Results: The results highlight the importance and value of bioimaging RDM and data sharing. However, the practical implementation of FAIR practices is impeded by technical hurdles, lack of knowledge, and insecurity about the legal aspects of data sharing. The survey participants request metadata guidelines and annotation tools and endorse the usage of image data management platforms. At present, OMERO (Open Microscopy Environment Remote Objects) is the best known and most widely used platform. Most respondents rely on image processing and analysis, which they regard as the most time-consuming step of the bioimage data workflow. While knowledge about and implementation of electronic lab notebooks and data management plans is limited, respondents acknowledge their potential value for data handling and publication. Conclusion: The bioimaging community acknowledges and endorses the value of RDM and data sharing. Still, there is a need for information, guidance, and standardization to foster the adoption of FAIR data handling. This survey may help inspiring targeted measures to close this gap.

The Shared Core Resource as a Partner in Innovative Scientific Research: Illustration from an Academic Microscopy Imaging Center

Core facilities have a ubiquitous and increasingly valuable presence at research institutions. Although many shared cores were originally created to provide routine services and access to complex and expensive instrumentation for the research community, they are frequently called upon by investigators to design protocols and procedures to help answer complex research questions. For instance, shared microscopy resources are evolving from providing access to and training on complex imaging instruments to developing detailed innovative protocols and experimental strategies, including sample preparation techniques, staining, complex imaging parameters, and high-level image analyses. These approaches require close intellectual collaboration between core staff and research investigators to formulate and coordinate plans for protocol development suited to the research question. Herein, we provide an example of such coordinated collaboration between a shared microscopy facility and a team of scientists and clinician-investigators to approach a complex multiprobe immunostaining, imaging, and image analysis project investigating the tumor microenvironment from human breast cancer samples. Our hope is that this example may be used to convey to institute administrators the critical importance of the intellectual contributions of the scientific staff in core facilities to research endeavors.

Building a Quality Management System in a Core Facility: A Genomics Core Case Study

Core facilities are key resources supporting the academic research enterprise, providing access to innovative and essential technologies and expertise. Given the constraints placed on core facilities as recharge centers and the ever-changing research environment, an important competitive differentiator that can support rigorous and reproducible approaches in core labs is the implementation of a quality management system (QMS). This paper describes a systematic approach to building a QMS in a genomics core facility at the University of North Carolina School of Medicine. This model is based on principles of the International Organization for Standardization 9001 system with initiatives focused on process mapping, training (communication, customer service, performance management, development of standard operating procedures, and quality audits), root cause analysis, visual control boards, mock quality audits, and continuous improvement through metrics tracking and "voice of the customer" exercises. The goal of this paper is to share practical step-by-step recommendations and outcomes of this core facility QMS that are generally applicable to academic core facilities, regardless of technical focus. Application of these good laboratory practice principles will foster "competitiveness through compliance" and promote outstanding interdisciplinary research between academic cores and their nonacademic pharmaceutical and federal research partners. Additionally, implementation of the QMS qualified this core to apply for federally funded contracts, thereby diversifying its types of projects and sources of revenue.

Understanding experiments and research practices for reproducibility: an exploratory study

Scientific experiments and research practices vary across disciplines. The research practices followed by scientists in each domain play an essential role in the understandability and reproducibility of results. The "Reproducibility Crisis", where researchers find difficulty in reproducing published results, is currently faced by several disciplines. To understand the underlying problem in the context of the reproducibility crisis, it is important to first know the different research practices followed in their domain and the factors that hinder reproducibility. We performed an exploratory study by conducting a survey addressed to researchers representing a range of disciplines to understand scientific experiments and research practices for reproducibility. The survey findings identify a reproducibility crisis and a strong need for sharing data, code, methods, steps, and negative and positive results. Insufficient metadata, lack of publicly available data, and incomplete information in study methods are considered to be the main reasons for poor reproducibility. The survey results also address a wide number of research questions on the reproducibility of scientific results. Based on the results of our explorative study and supported by the existing published literature, we offer general recommendations that could help the scientific community to understand, reproduce, and reuse experimental data and results in the research data lifecycle.

A survey of research quality in core facilities

Core facilities are an effective way of making expensive experimental equipment available to a large number of researchers, and are thus well placed to contribute to efforts to promote good research practices. Here we report the results of a survey that asked core facilities in Europe about their approaches to the promotion of good research practices, and about their interactions with users from the first contact to the publication of the results. Based on 253 responses we identified four ways that good research practices could be encouraged: (i) motivating users to follow the advice and procedures for best research practice; (ii) providing clear guidance on data-management practices; (iii) improving communication along the whole research process; and (iv) clearly defining the responsibilities of each party.

The ABCs of reproducibility: effecting attitudes, behaviors and change to realize reproducibility

New tools and increased awareness are yet to enact meaningful changes in reproducibility. Tristan Free investigates why and explores promising initiatives.

A Review of the Scientific Rigor, Reproducibility, and Transparency Studies Conducted by the ABRF Research Groups

Shared research resource facilities, also known as core laboratories (Cores), are responsible for generating a significant and growing portion of the research data in academic biomedical research institutions. Cores represent a central repository for institutional knowledge management, with deep expertise in the strengths and limitations of technology and its applications. They inherently support transparency and scientific reproducibility by protecting against cognitive bias in research design and data analysis, and they have institutional responsibility for the conduct of research (research ethics, regulatory compliance, and financial accountability) performed in their Cores. The Association of Biomolecular Resource Facilities (ABRF) is a FASEB-member scientific society whose members are scientists and administrators that manage or support Cores. The ABRF Research Groups (RGs), representing expertise for an array of cutting-edge and established technology platforms, perform multicenter research studies to determine and communicate best practices and community-based standards. This review provides a summary of the contributions of the ABRF RGs to promote scientific rigor and reproducibility in Cores from the published literature, ABRF meetings, and ABRF RGs communications.