Translating a Culture of Quality to Clinical Research Conduct: Expanding the Clinical Development Quality Framework

The International Council on Harmonisation E8 Guidance Revision 1 (ICH E8(R1)) calls for creating a Culture of Quality that "values and rewards critical thinking and open, proactive dialogue about what is critical to quality." Across the biopharma landscape, clinical sites, sponsors, and service providers are working to translate this far-reaching guideline into working practices. This manuscript deconstructs key elements that comprise the critical thinking and open, proactive Culture of Quality "enablers." In addition, maturity models are provided so readers can visualize what a Culture of Quality looks like in their clinical research organization. These provide examples of high performing cultures of quality and useful tools for teams or organizations to measure and evolve their respective quality cultures.

Measurement Invariance in Intellectual and Developmental Disability Research

Measurement invariance (MI) is a psychometric property of an instrument indicating the degree to which scores from an instrument are comparable across groups. In recent years, there has been a marked uptick in publications using MI in intellectual and developmental disability (IDD) samples. Our goal here is to provide an overview of why MI is important to IDD researchers and to describe some challenges to evaluating it, with an eye towards nudging our subfield into a more thoughtful and measured interpretation of studies using MI.

Research integrity and academic medicine: the pressure to publish and research misconduct

CONTEXT

This narrative review article explores research integrity and the implications of scholarly work in medical education. The paper describes how the current landscape of medical education emphasizes research and scholarly activity for medical students, resident physicians, and faculty physician educators. There is a gap in the existing literature that fully explores research integrity, the challenges surrounding the significant pressure to perform scholarly activity, and the potential for ethical lapses by those involved in medical education.

OBJECTIVES

The objectives of this review article are to provide a background on authorship and publication safeguards, outline common types of research misconduct, describe the implications of publication in medical education, discuss the consequences of ethical breaches, and outline possible solutions to promote research integrity in academic medicine.

METHODS

To complete this narrative review, the authors explored the current literature utilizing multiple databases beginning in June of 2021, and they completed the literature review in January of 2023. To capture the wide scope of the review, numerous searches were performed. A number of Medical Subject Headings (MeSH) terms were utilized to identify relevant articles. The MeSH terms included "scientific misconduct," "research misconduct," "authorship," "plagiarism," "biomedical research/ethics," "faculty, medical," "fellowships and scholarships," and "internship and residency." Additional references were accessed to include medical school and residency accreditation standards, residency match statistics, regulatory guidelines, and standard definitions.

RESULTS

Within the realm of academic medicine, research misconduct and misrepresentation continue to occur without clear solutions. There is a wide range of severity in breaches of research integrity, ranging from minor infractions to fraud. Throughout the medical education system in the United States, there is pressure to publish research and scholarly work. Higher rates of publications are associated with a successful residency match for students and academic promotion for faculty physicians. For those who participate in research misconduct, there is a multitude of potential adverse consequences. Potential solutions to ensure research integrity exist but are not without barriers to implementation.

CONCLUSIONS

Pressure in the world of academic medicine to publish contributes to the potential for research misconduct and authorship misrepresentation. Lapses in research integrity can result in a wide range of potentially adverse consequences for the offender, their institution, the scientific community, and the public. If adopted, universal research integrity policies and procedures could make major strides in eliminating research misconduct in the realm of academic medicine.

Implementation of the EQIPD Quality System

The EQIPD Quality System (QS) was conceptualized and established by an international consortium consisting of academic and industrial partners to ensure that non-regulated biomedical research will be conducted according to Good Research Practice expectations. The QS supports researchers to reflect on and improve internal practices by providing a systematic framework and guidance for implementing the EQIPD QS in a time and cost effective manner. This report describes the content of the EQIPD QS with its key features and 18 Core Requirements (CR) in more detail. It gives a short background on each CR and hands on examples on how they were addressed by two different research labs in their respective laboratory environments. Thereby, this article provides examples and direction for other research labs who aim to implement the QS as well. The final paragraphs discuss the potential benefits of the QS in respect to different user groups and stakeholders within the scientific community and summarize the overall governance structure of the EQIPD framework.

Virology-the path forward

In the United States (US), biosafety and biosecurity oversight of research on viruses is being reappraised. Safety in virology research is paramount and oversight frameworks should be reviewed periodically. Changes should be made with care, however, to avoid impeding science that is essential for rapidly reducing and responding to pandemic threats as well as addressing more common challenges caused by infectious diseases. Decades of research uniquely positioned the US to be able to respond to the COVID-19 crisis with astounding speed, delivering life-saving vaccines within a year of identifying the virus. We should embolden and empower this strength, which is a vital part of protecting the health, economy, and security of US citizens. Herein, we offer our perspectives on priorities for revised rules governing virology research in the US.

Coproducing health research with Indigenous peoples

The coproduction of health research represents an important advance in the realm of participatory methodologies, which have evolved over the past five decades. This transition to a collaborative approach emphasizes shared control between academic researchers and their partners, fostering a more balanced influence on the research process. This shift not only enhances the quality of the research and the evidence generated, but also increases the likelihood of successful implementation. For Indigenous peoples, coproduced research represents a critical development, enabling a shift from being mere 'subjects' of research to being active controllers of the process-including addressing the extractive and oppressive practices of the past. In this Review, we explore how research coproduction with Indigenous peoples is evolving. An 'Indigenous turn' embraces the concept of shared control while also considering the principles of reciprocity, the incommensurability of Western and Indigenous knowledge systems, divergent ethical standards, strategic and political differences, and the broader impact of processes and outcomes. To illustrate these ideas, we present examples involving New Zealand's Māori communities and offer recommendations for further progress.

Promoting Data Harmonization of COVID-19 Research in Pregnant and Pediatric Populations

This Viewpoint investigates the use of common data elements to promote data harmonization in COVID-19–related studies of pediatric and pregnant populations.

Inquiry in psychology

When they discuss the Danish academic situation, Szulevicz, Lund and Lund (2021) address three questions about the academic training of psychology researchers: (a) why do Danish master's students in psychology more frequently choose the qualitative method for their research?; (b) what are psychology students working on?; and (c) what are they interested in? These three questions have led us to reflect on researcher training and the political and educational model universities adopt for psychology master's courses, not only in the Danish context, but also in other general contexts. In this commentary, we will discuss one strictly normative issue: what should the scientific ideal be for training researchers in psychology? Or more accurately: how can psychology contribute to discussions about the scientific ideal of researcher training in this knowledge area?

Mississippi INBRE Outreach Scholars program: adapting a summer scholars program to the virtual world amidst the COVID-19 pandemic

The Mississippi IDeA Networks of Biomedical Research Excellence (INBRE) supported by the National Institute of General Medical Sciences (Grant P20GM103476) launched the new Mississippi INBRE Outreach Scholars (MIOS) summer research program in 2019. The program was designed to offer students community outreach and research experiences related to the study of behavioral and health disparities life sciences. The program was adapted in early 2020 to offer the program in a fully online format in the summer of 2020. This article details the program adaptations and discusses program evaluation data related to scholars' perceptions of program benefits and expectations and their confidence in research-related skills. The program evaluation was a mixed-method approach that included a qualitative postprogram survey and a pre-post quantitative survey. Scholars identified technical and communication skill building and resilience as areas of personal growth. Overall, the program met scholars' expectations for the program and significantly improved their confidence on 8 of the 19 (with confidence interval estimated differences from 0.3 to 2.56, where a difference of 1 is an improvement across 1 anchor on a Likert-type scale) various research-related tasks/skills after completion of the program. The analyses presented demonstrated that a combined qualitative and quantitative analysis approach is useful for examining the extent to which programs such as Mississippi INBRE are meeting goals of providing a rich research experience in health disparities for a diverse student body. Future longitudinal data may be examined to explore the long-term impact of MIOS on career preparation and choices and graduate education.NEW & NOTEWORTHY The Mississippi INBRE Outreach Scholars program is a summer research program for Mississippi college students that was successfully adapted to a fully online environment amidst the coronavirus-19 pandemic.

Reported Interest in Notification Regarding Use of Health Information and Biospecimens

This study describes reported interest in notification regarding use of personal health information and biospecimens for research and preference-associated factors among a sample of the US population in 2019.

Reproducibility, relevance and reliability as barriers to efficient and credible biomedical technology translation

Biomedical research accuracy and relevance for improving healthcare are increasingly identified as costly problems. Basic research data quality, reporting and methodology, and reproducibility are common factors implicated in this challenge. Preclinical models of disease and therapy, largely conducted in rodents, have known deficiencies in replicating most human conditions. Their translation to human results is acknowledged to be poor for decades. Clinical data quality and quantity is also recognized as deficient; gold standard randomized clinical trials are expensive. Few solid conclusions from clinical studies are replicable and many remain unpublished. The translational pathway from fundamental biomedical research through to innovative solutions handed to clinical practitioners is therefore highly inefficient and costly in terms of wasted resources, early claims from fundamental discoveries never witnessed in humans, and few new, improved solutions available clinically for myriad diseases. Improving this biomedical research strategy and resourcing for reliability, translational relevance, reproducibility and clinical impact requires careful analysis and consistent enforcement at both funding and peer review levels.

ENIGMA-DTI: Translating reproducible white matter deficits into personalized vulnerability metrics in cross-diagnostic psychiatric research

The ENIGMA-DTI (diffusion tensor imaging) workgroup supports analyses that examine the effects of psychiatric, neurological, and developmental disorders on the white matter pathways of the human brain, as well as the effects of normal variation and its genetic associations. The seven ENIGMA disorder-oriented working groups used the ENIGMA-DTI workflow to derive patterns of deficits using coherent and coordinated analyses that model the disease effects across cohorts worldwide. This yielded the largest studies detailing patterns of white matter deficits in schizophrenia spectrum disorder (SSD), bipolar disorder (BD), major depressive disorder (MDD), obsessive-compulsive disorder (OCD), posttraumatic stress disorder (PTSD), traumatic brain injury (TBI), and 22q11 deletion syndrome. These deficit patterns are informative of the underlying neurobiology and reproducible in independent cohorts. We reviewed these findings, demonstrated their reproducibility in independent cohorts, and compared the deficit patterns across illnesses. We discussed translating ENIGMA-defined deficit patterns on the level of individual subjects using a metric called the regional vulnerability index (RVI), a correlation of an individual's brain metrics with the expected pattern for a disorder. We discussed the similarity in white matter deficit patterns among SSD, BD, MDD, and OCD and provided a rationale for using this index in cross-diagnostic neuropsychiatric research. We also discussed the difference in deficit patterns between idiopathic schizophrenia and 22q11 deletion syndrome, which is used as a developmental and genetic model of schizophrenia. Together, these findings highlight the importance of collaborative large-scale research to provide robust and reproducible effects that offer insights into individual vulnerability and cross-diagnosis features.

Consolidated Health Economic Evaluation Reporting Standards (CHEERS) 2022 Explanation and Elaboration: A Report of the ISPOR CHEERS II Good Practices Task Force

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces the previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, and the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as healthcare, public health, education, and social care). This Explanation and Elaboration Report presents the new CHEERS 2022 28-item checklist with recommendations and explanation and examples for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer-reviewed journals and the peer reviewers and editors assessing them for publication. Nevertheless, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, given that there is an increasing emphasis on transparency in decision making.

Compliance of Clinical Trial Protocols for Foods with Function Claims (FFC) in Japan: Consistency between Clinical Trial Registrations and Published Reports

Background: A new type of foods with a health claims notification system, the Foods with Function Claims (FFC), was introduced in Japan in April 2015. This cross-sectional study sought to clarify compliance of clinical trial protocols reported as the scientific basis of efficacy in the FFC system. Methods: All articles based on clinical trials published on the Consumer Affairs Agency website from 1 July 2018 to 30 June 2021 were reviewed. Items assessed included first author characteristics (for-profit or academia), journal name, year published, journal impact factor in 2020, article language, name of clinical trial registration (CTR), and seven compliance items (Title: T, Participant: P, Intervention: I, Comparison: C, Outcome: O, Study design: S, and Institutional Review Board, IRB). Among studies that conducted CTR, consistency with these seven compliance items was evaluated. Results: Out of 136 studies that met all inclusion criteria, 103 (76%) performed CTR, and CTR was either not performed or not specified for 33 (24%). Compliance between the protocol and the text was high (≥96%) for items P and S, but considerably lower for items T, I, C, O, and IRB (52%, 15%, 13%, 69%, and 27%, respectively). Furthermore, 43% of protocols did not include functional ingredients or food names in items T or I. The total score was 3.7 ± 1.1 pts (out of 7). Conclusions: Some CTs had no protocol registration, and even registered protocols were suboptimal in transparency. In addition to selective reporting, a new problem identified was that the content of the intervention (test food) was intentionally concealed.

Guidelines for in vivo mouse models of myocardial infarction

Despite significant improvements in reperfusion strategies, acute coronary syndromes all too often culminate in a myocardial infarction (MI). The consequent MI can, in turn, lead to remodeling of the left ventricle (LV), the development of LV dysfunction, and ultimately progression to heart failure (HF). Accordingly, an improved understanding of the underlying mechanisms of MI remodeling and progression to HF is necessary. One common approach to examine MI pathology is with murine models that recapitulate components of the clinical context of acute coronary syndrome and subsequent MI. We evaluated the different approaches used to produce MI in mouse models and identified opportunities to consolidate methods, recognizing that reperfused and nonreperfused MI yield different responses. The overall goal in compiling this consensus statement is to unify best practices regarding mouse MI models to improve interpretation and allow comparative examination across studies and laboratories. These guidelines will help to establish rigor and reproducibility and provide increased potential for clinical translation.

The importance of cell culture parameter standardization: an assessment of the robustness of the 2102Ep reference cell line

Work undertaken using the embryonic carcinoma 2102Ep line, highlighted the requirement for robust, well-characterized and standardized protocols. A systematic approach utilizing 'quick hit' experiments demonstrated variability introduced into culture systems resulting from slight changes to culture conditions (route A). This formed the basis for longitudinal experiments investigating long-term effects of culture parameters including seeding density and feeding regime (route B).Results demonstrated that specific growth rates (SGR) of passage 59 (P59) cells seeded at 20,000 cells/cm2 and subjected to medium exchange after 48h prior to reseeding at 72h (route B2) on average was marginally higher than, P55 cells cultured under equivalent conditions (route A1); whereby SGR values were (0.021±0.004) and (0.019±0.004). Viability was higher in route B2 over 10 passages with average viability reported as (86.3%±8.1) compared to route A1 (83.3±8.8). The metabolite data demonstrated both culture route B1 (P57 cells seeded at 66,667 cells/cm2) and B2 had consistent-specific metabolite rates (SMR) for glucose, but SMR values of route B1 was consistently lower than route B2 (0.00001 mmol, cell-1.d-1 and 0.000025).Results revealed interactions between phenotype, SMR and feeding regime that may not be accurately reflected by growth rate or observed morphology. This implies that current schemes of protocol control do not adequately account for variability, since key cell characteristics, including phenotype and SMR, change regardless of standardized seeding densities. This highlights the need to control culture parameters through defined protocols, for processes that involve culture for therapeutic use, biologics production, and reference lines.

Graphical integrity issues in open access publications: Detection and patterns of proportional ink violations

Academic graphs are essential for communicating complex scientific ideas and results. To ensure that these graphs truthfully reflect underlying data and relationships, visualization researchers have proposed several principles to guide the graph creation process. However, the extent of violations of these principles in academic publications is unknown. In this work, we develop a deep learning-based method to accurately measure violations of the proportional ink principle (AUC = 0.917), which states that the size of shaded areas in graphs should be consistent with their corresponding quantities. We apply our method to analyze a large sample of bar charts contained in 300K figures from open access publications. Our results estimate that 5% of bar charts contain proportional ink violations. Further analysis reveals that these graphical integrity issues are significantly more prevalent in some research fields, such as psychology and computer science, and some regions of the globe. Additionally, we find no temporal and seniority trends in violations. Finally, apart from openly releasing our large annotated dataset and method, we discuss how computational research integrity could be part of peer-review and the publication processes.

Downstream funding success of early career researchers for resubmitted versus new applications: A matched cohort

Background

Early career researchers face a hypercompetitive funding environment. To help identify effective intervention strategies for early career researchers, we examined whether first-time NIH R01 applicants who resubmitted their original, unfunded R01 application were more successful at obtaining any R01 funding within 3 and 5 years than original, unfunded applicants who submitted new NIH applications, and we examined whether underrepresented minority (URM) applicants differentially benefited from resubmission. Our observational study is consistent with an NIH working group’s recommendations to develop interventions to encourage resubmission.

Methods and findings

First-time applicants with US medical school academic faculty appointments who submitted an unfunded R01 application between 2000–2014 yielded 4,789 discussed and 7,019 not discussed applications. We then created comparable groups of first-time R01 applicants (resubmitted original R01 application or submitted new NIH applications) using optimal full matching that included applicant and application characteristics. Primary and subgroup analyses used generalized mixed models with obtaining any NIH R01 funding within 3 and 5 years as the two outcomes. A gamma sensitivity analysis was performed. URM applicants represented 11% and 12% of discussed and not discussed applications, respectively.

First-time R01 applicants resubmitting their original, unfunded R01 application were more successful obtaining R01 funding within 3 and 5 years than applicants submitting new applications—for both discussed and not discussed applications: discussed within 3 years (OR 4.17 [95 CI 3.53, 4.93]) and 5 years (3.33 [2.82–3.92]); and not discussed within 3 years (2.81 [2.52, 3.13]) and 5 years (2.47 [2.22–2.74]). URM applicants additionally benefited within 5 years for not discussed applications.

Conclusions

Encouraging early career researchers applying as faculty at a school of medicine to resubmit R01 applications is a promising potential modifiable factor and intervention strategy. First-time R01 applicants who resubmitted their original, unfunded R01 application had log-odds of obtaining downstream R01 funding within 3 and 5 years 2–4 times higher than applicants who did not resubmit their original application and submitted new NIH applications instead. Findings held for both discussed and not discussed applications.

More consideration is needed for retracted non-Cochrane systematic reviews in medicine: a systematic review

OBJECTIVE

To analyze the retraction status and reasons of non-Cochrane systematic reviews (SRs) in medicine.

STUDY DESIGN AND SETTING

MEDLINE, Embase, Retraction Watch Database and Google Scholar were systematically searched to find all retracted non-Cochrane SRs.

RESULTS

Of 159 non-Cochrane SRs in medicine retracted between 2004 and 2020, more than 70% were led by authors from China and affiliated with hospitals. The largest proportion of retraction notices were issued by the publisher and editor(s) jointly. Fraudulent peer-review was the most common reason for retraction, followed by unreliable data meaning errors in study selection or data analysis. The median time between publication and retraction was 14 months, and SRs retracted due to research misconduct took longer to retract than honest error.

CONCLUSION

The total number of retracted SRs is increasing worldwide, in particular in China. The most common reasons for retraction are fraudulent peer-review and unreliable data, and in most cases the SR is retracted more than a year after publication. Better systems of ethical oversight and culture to improve the process of peer review and adherence to the COPE retraction guidance are needed, and authors should strengthen their skills in SR methodology.

Reporting of Adverse Events in Randomized Controlled Trials of Therapeutic Exercise for Hip Osteoarthritis: A Systematic Review

OBJECTIVE

The purpose of this study was to describe adverse events (AEs) and dropouts (DOs) in randomized controlled trials of therapeutic exercise for hip osteoarthritis (HOA) and to identify whether Consolidated Standards of Reporting Trials (CONSORT) guidelines were followed.

METHODS

The Cochrane Library, Embase, PubMed, and CINAHL databases were searched. Randomized controlled trials of therapeutic exercise for HOA published in English from January 1, 1980 to August 1, 2020 were included. Studies were excluded if other interventions were provided, if participants had previous hip arthroplasty, or if AEs and DOs for HOA participants were not reported separately. The internal validity of each study (Physiotherapy Evidence Database [PEDro] scoring) was assessed, participant and intervention characteristics were extracted, and the existence of a clear statement and reasons for AEs and DOs was reported. Descriptive statistics characterized results. Data heterogeneity prohibited the use of meta-analysis.

RESULTS

Fourteen studies (mean PEDro score = 7.4; range = 6-10) from 10 countries were included, with 707 participants exercising. Exercise intensity was unspecified in 72.2% of exercise arms. Six studies (42.9%) included a statement of AEs, and 32 AEs were reported. All studies had a DO statement, but 29.0% of DOs occurred for unknown reasons. Six studies (42.9%) gave reasons for DOs that could be classified as AEs in 9 participants; 41 participants (5.8%) experienced exercise-related AEs.

CONCLUSION

Reports of AEs were inconsistent, some DOs were potentially misclassified, and primary components of exercise interventions were frequently unreported. Despite these limitations, the overall low number of nonserious AEs suggests that the exercise-related risk of harm is minimal for individuals with HOA.

IMPACT

Understanding the risk of harm associated with exercise for HOA can better inform safe dosing of exercise, clinical implementation, and replicability. Informative, consistent reporting of AEs, DOs, and exercise is needed. Greater use of the CONSORT harms-reporting checklist is warranted.

Barriers and Facilitating Factors for Conducting Systematic Evidence Assessments in Academic Clinical Trials

IMPORTANCE

A systematic assessment of existing research should justify the conduct and inform the design of new clinical research but is often lacking. There is little research on the barriers to and factors facilitating systematic evidence assessments.

OBJECTIVE

To examine the practices and attitudes of Swiss stakeholders and international funders regarding conducting systematic evidence assessments in academic clinical trials.

DESIGN, SETTING, AND PARTICIPANTS

In this qualitative study, individual semistructured qualitative interviews were conducted between February and August 2020 with 48 Swiss stakeholder groups (27 primary investigators, 9 funders and sponsors, 6 clinical trial support organizations, and 6 ethics committee members) and between January and March 2021 with 9 international funders of clinical trials from North America and Europe with a reputation for requiring systematic evidence synthesis in applications for academic clinical trials.

MAIN OUTCOMES AND MEASURES

The main outcomes were practices and attitudes of Swiss stakeholders and international funders regarding conducting systematic evidence assessments in academic clinical trials. Interviews were analyzed using conventional content analysis.

RESULTS

Of the 57 participants, 40 (70.2%) were male. Participants universally acknowledged that a comprehensive understanding of the previous evidence is important but reported wide variation regarding how this should be achieved. Participants reported that the conduct of formal systematic reviews was currently not expected before most clinical trials, but most international funders reported expecting a systematic search for the existing evidence. Whereas time and resources were reported by all participants as barriers to conducting systematic reviews, the Swiss research ecosystem was reported not to be as supportive of a systematic approach compared with international settings.

CONCLUSIONS AND RELEVANCE

In this qualitative study, Swiss stakeholders and international funders generally agreed that new clinical trials should be justified by a systematic evidence assessment but that barriers on individual, organizational, and political levels kept them from implementing it. More explicit requirements from funders appear to be needed to clarify the required level of comprehensiveness in summarizing existing evidence for different types of clinical trials.

COVID-19 related interdisciplinary methods: Preventing errors and detecting research opportunities

More than 130,000 peer-reviewed studies have been published within one year after COVID-19 emerged in many countries. This large and rapidly growing field may overwhelm the synthesizing abilities of both researchers and policy-makers. To provide a sinopsis, prevent errors, and detect cognitive gaps that may require interdisciplinary research methods, the literature on COVID-19 is summarized, twice. The overall purpose of this study is to generate a dialogue meant to explain the genesis of and/or find remedies for omissions and contradictions. The first review starts in Biology and ends in Policy. Policy is chosen as a destination because it is the setting where cognitive integration must occur. The second review follows the opposite path: it begins with stated policies on COVID-19 and then their assumptions and disciplinary relationships are identified. The purpose of this interdisciplinary method on methods is to yield a relational and explanatory view of the field -one strategy likely to be incomplete but usable when large bodies of literature need to be rapidly summarized. These reviews identify nine inter-related problems, research needs, or omissions, namely: (1) nation-wide, geo-referenced, epidemiological data collection systems (open to and monitored by the public); (2) metrics meant to detect non-symptomatic cases -e.g., test positivity-; (3) cost-benefit oriented methods, which should demonstrate they detect silent viral spreaders even with limited testing; (4) new personalized tests that inform on biological functions and disease correlates, such as cell-mediated immunity, co-morbidities, and immuno-suppression; (5) factors that influence vaccine effectiveness; (6) economic predictions that consider the long-term consequences likely to follow epidemics that growth exponentially; (7) the errors induced by self-limiting and/or implausible paradigms, such as binary and reductionist approaches; (8) new governance models that emphasize problem-solving skills, social participation, and the use of scientific knowledge; and (9) new educational programs that utilize visual aids and audience-specific communication strategies. The analysis indicates that, to optimally address these problems, disciplinary and social integration is needed. By asking what is/are the potential cause(s) and consequence(s) of each issue, this methodology generates visualizations that reveal possible relationships as well as omissions and contradictions. While inherently limited in scope and likely to become obsolete, these shortcomings are avoided when this 'method on methods' is frequently practiced. Open-ended, inter-/trans-disciplinary perspectives and broad social participation may help researchers and citizens to construct, de-construct, and re-construct COVID-19 related research.

Resource use during systematic review production varies widely: a scoping review

OBJECTIVE

We aimed to map the resource use during systematic review (SR) production and reasons why steps of the SR production are resource intensive to discover where the largest gain in improving efficiency might be possible.

STUDY DESIGN AND SETTING

We conducted a scoping review. An information specialist searched multiple databases (e.g., Ovid MEDLINE, Scopus) and implemented citation-based and grey literature searching. We employed dual and independent screenings of records at the title/abstract and full-text levels and data extraction.

RESULTS

We included 34 studies. Thirty-two reported on the resource use-mostly time; four described reasons why steps of the review process are resource intensive. Study selection, data extraction, and critical appraisal seem to be very resource intensive, while protocol development, literature search, or study retrieval take less time. Project management and administration required a large proportion of SR production time. Lack of experience, domain knowledge, use of collaborative and SR-tailored software, and good communication and management can be reasons why SR steps are resource intensive.

CONCLUSION

Resource use during SR production varies widely. Areas with the largest resource use are administration and project management, study selection, data extraction, and critical appraisal of studies.

Toward an Anti-Racist Approach to Biomedical and Neuroscience Research

Racism is a threat to public health. Race is a sociopolitical construct that has been used for generations to create disparities in educational access, housing conditions, exposure to environmental contaminants, and access to health care. Collectively, these disparities have a negative impact on the health of non-white Americans. The National Institutes of Health (NIH) funds biomedical research, including basic neuroscience research, aimed at understanding the mechanisms and consequences of health and disease in Americans. NIH has recently acknowledged its own structural racism, the disadvantage this perpetuates in the biomedical research enterprise, and has announced its commitment to eliminating these disparities. Here, we discuss different rates of disease in U.S. citizens from different racial backgrounds. We next describe ways in which the biomedical research enterprise (1) has contributed to health disparities and (2) can contribute to the solving this problem. Based on our own scientific expertise, we use neuroscience in general and mental health/addiction disorders more specifically as examples of a broader issue. The NIH, including its neuroscience-focused Institutes, and NIH-funded scientists, including neuroscientists, should prioritize research topics that reflect the health conditions that affect all Americans, not just white Americans.

Avoiding a replication crisis in deep-learning-based bioimage analysis

Deep learning algorithms are powerful tools to analyse, restore and transform bioimaging data, increasingly used in life sciences research. These approaches now outperform most other algorithms for a broad range of image analysis tasks. In particular, one of the promises of deep learning is the possibility to provide parameter-free, one-click data analysis achieving expert-level performances in a fraction of the time previously required. However, as with most new and upcoming technologies, the potential for inappropriate use is raising concerns among the biomedical research community. This perspective aims to provide a short overview of key concepts that we believe are important for researchers to consider when using deep learning for their microscopy studies. These comments are based on our own experience gained while optimising various deep learning tools for bioimage analysis and discussions with colleagues from both the developer and user community. In particular, we focus on describing how results obtained using deep learning can be validated and discuss what should, in our views, be considered when choosing a suitable tool. We also suggest what aspects of a deep learning analysis would need to be reported in publications to describe the use of such tools to guarantee that the work can be reproduced. We hope this perspective will foster further discussion between developers, image analysis specialists, users and journal editors to define adequate guidelines and ensure that this transformative technology is used appropriately.

Reporting of prognostic clinical prediction models based on machine learning methods in oncology needs to be improved

OBJECTIVE

Evaluate the completeness of reporting of prognostic prediction models developed using machine learning methods in the field of oncology.

STUDY DESIGN AND SETTING

We conducted a systematic review, searching the MEDLINE and Embase databases between 01/01/2019 and 05/09/2019, for non-imaging studies developing a prognostic clinical prediction model using machine learning methods (as defined by primary study authors) in oncology. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement to assess the reporting quality of included publications. We described overall reporting adherence of included publications and by each section of TRIPOD.

RESULTS

Sixty-two publications met the inclusion criteria. 48 were development studies and 14 were development with validation studies. 152 models were developed across all publications. Median adherence to TRIPOD reporting items was 41% [range: 10%-67%] and at least 50% adherence was found in 19% (n=12/62) of publications. Adherence was lower in development only studies (median: 38% [range: 10%-67%]); and higher in development with validation studies (median: 49% [range: 33%-59%]).

CONCLUSION

Reporting of clinical prediction models using machine learning in oncology is poor and needs urgent improvement, so readers and stakeholders can appraise the study methods, understand study findings, and reduce research waste.

Cluster randomized trials of individual-level interventions were at high risk of bias

OBJECTIVES

To describe the prevalence of risks of bias in cluster-randomized trials of individual-level interventions, according to the Cochrane Risk of Bias tool.

STUDY DESIGN AND SETTING

Review undertaken in duplicate of a random sample of 40 primary reports of cluster-randomized trials of individual-level interventions.

RESULTS

The most common reported reasons for adopting cluster randomization were the need to avoid contamination (17, 42.5%) and practical considerations (14, 35%). Of the 40 trials all but one was assessed as being at risk of bias. A majority (27, 67.5%) were assessed as at risk due to the timing of identification and recruitment of participants; many (21, 52.5%) due to an apparent lack of adequate allocation concealment; and many due to selectively reported results (22, 55%), arising from a mixture of reasons including lack of documentation of primary outcome. Other risks mostly occurred infrequently.

CONCLUSION

Many cluster-randomized trials evaluating individual-level interventions appear to be at risk of bias, mostly due to identification and recruitment biases. We recommend that investigators carefully consider the need for cluster randomization; follow recommended procedures to mitigate risks of identification and recruitment bias; and adhere to good reporting practices including clear documentation of primary outcome and allocation concealment methods.

Methodology over metrics: current scientific standards are a disservice to patients and society

Covid-19 research made it painfully clear that the scandal of poor medical research, as denounced by Altman in 1994, persists today. The overall quality of medical research remains poor, despite longstanding criticisms. The problems are well known, but the research community fails to properly address them. We suggest that most problems stem from an underlying paradox: although methodology is undeniably the backbone of high-quality and responsible research, science consistently undervalues methodology. The focus remains more on the destination (research claims and metrics) than on the journey. Notwithstanding, research should serve society more than the reputation of those involved. While we notice that many initiatives are being established to improve components of the research cycle, these initiatives are too disjointed. The overall system is monolithic and slow to adapt. We assert that top-down action is needed from journals, universities, funders and governments to break the cycle and put methodology first. These actions should involve the widespread adoption of registered reports, balanced research funding between innovative, incremental and methodological research projects, full recognition and demystification of peer review, improved methodological review of reports, adherence to reporting guidelines, and investment in methodological education and research. Currently, the scientific enterprise is doing a major disservice to patients and society.

Recommended reporting items for epidemic forecasting and prediction research: The EPIFORGE 2020 guidelines

BACKGROUND

The importance of infectious disease epidemic forecasting and prediction research is underscored by decades of communicable disease outbreaks, including COVID-19. Unlike other fields of medical research, such as clinical trials and systematic reviews, no reporting guidelines exist for reporting epidemic forecasting and prediction research despite their utility. We therefore developed the EPIFORGE checklist, a guideline for standardized reporting of epidemic forecasting research.

METHODS AND FINDINGS

We developed this checklist using a best-practice process for development of reporting guidelines, involving a Delphi process and broad consultation with an international panel of infectious disease modelers and model end users. The objectives of these guidelines are to improve the consistency, reproducibility, comparability, and quality of epidemic forecasting reporting. The guidelines are not designed to advise scientists on how to perform epidemic forecasting and prediction research, but rather to serve as a standard for reporting critical methodological details of such studies.

CONCLUSIONS

These guidelines have been submitted to the EQUATOR network, in addition to hosting by other dedicated webpages to facilitate feedback and journal endorsement.

Most ankle sprain research is either false or clinically unimportant: A 30-year audit of randomized controlled trials

BACKGROUND

Lateral ankle sprain is the most common musculoskeletal injury. Although clinical research in this field is growing, there is a broader concern that clinical trial outcomes are often false and fail to translate into patient benefits.

METHODS

We audited 30 years of experimental research related to lateral ankle sprain management (n = 74 randomized controlled trials) to determine if reports of treatment effectiveness could be validated beyond statistical certainty.

RESULTS

A total of 77% of trials reported positive treatment effects, but there was a high risk of false discovery. Most trials were unregistered and relied solely on statistical significance, or lack of statistical significance, rather than on interpreting key measures of minimum clinical importance (e.g., minimal detectable change, minimal clinically important difference).

CONCLUSION

Future clinical trials must adopt higher standards of reporting and data interpretation. This includes consideration of the ethical responsibility to preregister their research and interpretation of clinical outcomes beyond statistical significance.

Management of diabetes mellitus through teleconsultation during COVID-19 and similar scenarios - Guidelines from Indian Council of Medical Research (ICMR) expert group

INTRODUCTION

Emergence of COVID-19 pandemic has led to increased use of telemedicine in health care delivery. Telemedicine facilitates long-term clinical care for monitoring and prevention of complications of diabetes mellitus.

GUIDELINES

Precise indications for teleconsultation, clinical care services which can be provided, and good clinical practices to be followed during teleconsultation are explained. Guidance on risk assessment and health education for diabetes risk factors, counselling for blood glucose monitoring, treatment compliance, and prevention of complications are described.

CONCLUSION

The guidelines will help physicians in adopting teleconsultation for management of diabetes mellitus, facilitate access to diabetes care and improve health outcomes.

Methodological options of the nominal group technique for survey item elicitation in health research: A scoping review

OBJECTIVE

To conduct a scoping review that identifies different nominal group technique (NGT) methods used to elicit items for health surveys, and their advantages and disadvantages.

STUDY DESIGN AND SETTING

We conducted a comprehensive search process from database inception to July 22, 2019 in Medline, EMBASE, PsychInfo, CINAHL, Cochrane Central and Scopus without language restriction. We screened titles and abstracts. Data from potentially relevant articles were extracted by one reviewer and verified by a second reviewer, with disagreements resolved by consensus or a third reviewer.

RESULTS

We included 57 studies, which used between 1 and 41 nominal groups that included between 2 and 30 participants per group. We grouped the 30 identified decision points for the NGT process into two stages common to most qualitative group methods [Research objectives; Group characteristics] and three stages related to the nominal groups themselves [Eliciting survey items; Refining survey elicited items from stage 3; Evaluating and selecting final survey items]. We discuss the advantages and disadvantages of each option in relation to specific study contexts.

CONCLUSION

Investigators should carefully consider their options for each of the identified decision points and document the reasons for their choices in their protocol to maximize validity and transparency.

Predicting the number of article citations on the topic of pemphigus vulgaris with the 100 top-cited articles since 2011: A protocol for systematic review and meta-analysis

BACKGROUND

Pemphigus vulgaris (PV) is a rare autoimmune blistering disease characterized by intraepithelial and mucocutaneous blister formation and erosion. Numerous articles related to PV have been published. However, which articles have a tremendous influence is still unknown, and factors affecting article citation numbers remain unclear. We aimed to visualize the prominent entities using the top 100 most-cited articles on the topic of PV (T100PV), and investigate whether medical subject headings (i.e., MeSH terms) can be used to predict article citations.

METHODS

By searching the PubMed Central (PMC) database, the T100PV abstracts since 2011 were downloaded. Citation analysis was performed to compare the dominant entities in article topics, authors, and research institutes using social network analysis (SNA) and Kano diagrams. We examined the MeSH prediction power against article citations using correlation coefficients (CCs).

RESULTS

The most cited article (125 times) was authored by Ellebrecht from the University of Pennsylvania in the US. The most productive countries were Germany (28%) and the US (25%). Most articles were published in J Invest Dermatol (16%) and Br J Dermatol (10%). Kasperkiewicz (Germany) and the Normandie University (France) were the most cited authors and research institutes, respectively. The most frequently occurred MeSH terms were administration and dosage, immunology, and metabolism. MeSH terms were evident in the prediction power on the number of article citations (F = 19.77; P < .001).

CONCLUSION

A breakthrough was achieved by developing dashboards to display the T100PV. MeSH terms can be used to predict the T100PV citations. These T100PV visualizations can be applied in future studies.

Human sample authentication in biomedical research: comparison of two platforms

Samples used in biomedical research are often collected over years, in some cases from subjects that may have died and thus cannot be retrieved in any way. The value of these samples is priceless. Sample misidentification or mix-up are unfortunately common problems in biomedical research and can eventually result in the publication of incorrect data. Here we have compared the Fluidigm SNPtrace and the Agena iPLEX Sample ID panels for the authentication of human genomic DNA samples. We have tested 14 pure samples and simulated their cross-contamination at different percentages (2%, 5%, 10%, 25% and 50%). For both panels, we report call rate, allele intensity/probability score, performance in distinguishing pure samples and contaminated samples at different percentages, and sex typing. We show that both panels are reliable and efficient methods for sample authentication and we highlight their advantages and disadvantages. We believe that the data provided here is useful for sample authentication especially in biorepositories and core facility settings.

Pilot and feasibility studies for pragmatic trials have unique considerations and areas of uncertainty

BACKGROUND AND OBJECTIVE

Feasibility studies are increasingly being used to support the development of, and investigate uncertainties around, future large-scale trials. The future trial can be designed with either a pragmatic or explanatory mindset. Whereas pragmatic trials aim to inform the choice between different care options and thus, are designed to resemble conditions outside of a clinical trial environment, explanatory trials examine the benefit of a treatment under more controlled conditions. There is existing guidance for designing feasibility studies, but none that explicitly considers the goals of pragmatic designs. We aimed to identify unique areas of uncertainty that are relevant to planning a pragmatic trial.

RESULTS

We identified ten relevant domains, partly based on the pragmatic-explanatory continuum indicator summary-2 (PRECIS-2) framework, and describe potential questions of uncertainty within each: intervention development, research ethics, participant identification and eligibility, recruitment of individuals, setting, organization, flexibility of delivery, flexibility of adherence, follow-up, and importance of primary outcome to patients and decision-makers. We present examples to illustrate how uncertainty in these domains might be addressed within a feasibility study.

CONCLUSION

Researchers planning a feasibility study in advance of a pragmatic trial should consider feasibility objectives specifically relevant to areas of uncertainty for pragmatic trials.

Administrative Coding in Electronic Health Care Record-Based Research of NAFLD: An Expert Panel Consensus Statement

BACKGROUND AND AIMS

Electronic health record (EHR)-based research allows the capture of large amounts of data, which is necessary in NAFLD, where the risk of clinical liver outcomes is generally low. The lack of consensus on which International Classification of Diseases (ICD) codes should be used as exposures and outcomes limits comparability and generalizability of results across studies. We aimed to establish consensus among a panel of experts on ICD codes that could become the reference standard and provide guidance around common methodological issues.

APPROACH AND RESULTS

Researchers with an interest in EHR-based NAFLD research were invited to collectively define which administrative codes are most appropriate for documenting exposures and outcomes. We used a modified Delphi approach to reach consensus on several commonly encountered methodological challenges in the field. After two rounds of revision, a high level of agreement (>67%) was reached on all items considered. Full consensus was achieved on a comprehensive list of administrative codes to be considered for inclusion and exclusion criteria in defining exposures and outcomes in EHR-based NAFLD research. We also provide suggestions on how to approach commonly encountered methodological issues and identify areas for future research.

CONCLUSIONS

This expert panel consensus statement can help harmonize and improve generalizability of EHR-based NAFLD research.

Developing a research database of primary aldosteronism: rationale and baseline characteristics

BACKGROUND

Management of primary aldosteronism (PA) has become a research hotspot in the field of endocrinology. To obtain reliable research evidence, it is necessary to establish a high-quality PA research database.

METHODS

The establishment of PA research database involved two steps. Firstly, patients with confirmation of PA diagnosis between 1 Jan 2009 to 31 Aug 2019 at West China Hospital were identified and data were extracted. Secondly, patients with confirmatory testing for PA will be enrolled into a prospective cohort. Data will be prospectively collected based on the case report forms since 1 Sep 2019. We evaluated the quality of research database through assessment of quality of key variables.

RESULTS

Totally, 862 patients diagnosed as PA were identified, of which 507 patients who had positive confirmatory testing for PA were included into the retrospective database. Among 862 patients diagnosed as PA, the mean systolic blood pressure (SBP) was 156.1 (21.7) mmHg, mean diastolic blood pressure (DBP) was 97.2 (14.5) mmHg. Among included patients, the mean serum potassium level was 2.85 (IQR, (2.47-3.36) mmol/L, and the mean plasma aldosterone concentration (PAC) was 28.1 (IQR, 20.0-40.4) ng/dL. The characteristics of patients with positive confirmatory testing for PA were similar. Validation of data extracting and linking showed the accuracy were 100%. Evaluation of missing data showed that the completeness of BMI (95.9%), SBP (99.4%) and DBP (99.4%) were high.

CONCLUSION

Through integrating retrospective and prospective cohort of PA, a research database of PA with high quality and comprehensive data can be established. We anticipate that the research database will provide a high level of feasibility for management of PA in China.