Completeness of reporting of systematic reviews in the animal health literature: A meta-research study

A suggested data structure for transparent and repeatable reporting of bibliographic searching

Academic searching is integral to research activities: (1) searching to retrieve specific information, (2) to expand our knowledge iteratively, (3) and to collate a representative and unbiased selection of the literature. Rigorous searching methods are vital for reliable, repeatable and unbiased searches needed for these second and third forms of searches (exploratory and systematic searching, respectively) that form a core part of evidence syntheses. Despite the broad awareness of the importance of transparency in reporting search activities in evidence syntheses, the importance of searching has been highlighted only recently and has been the explicit focus of reporting guidance (PRISMA-S). Ensuring bibliographic searches are reported in a way that is transparent enough to allow for full repeatability or evaluation is challenging for a number of reasons. Here, we detail these reasons and provide for the first time a standardised data structure for transparent and comprehensive reporting of search histories. This data structure was produced by a group of international experts in informatics and library sciences. We explain how the data structure was produced and describe its components in detail. We also demonstrate its practical applicability in tools designed to support literature review authors and explain how it can help to improve interoperability across tools used to manage literature reviews. We call on the research community and developers of reference and review management tools to embrace the data structure to facilitate adequate reporting of academic searching in an effort to raise the standard of evidence syntheses globally.

Levels of Evidence, Quality Assessment, and Risk of Bias: Evaluating the Internal Validity of Primary Research

Clinical decisions in human and veterinary medicine should be based on the best available evidence. The results of primary research are an important component of that evidence base. Regardless of whether assessing studies for clinical case management, developing clinical practice guidelines, or performing systematic reviews, evidence from primary research should be evaluated for internal validity i.e., whether the results are free from bias (reflect the truth). Three broad approaches to evaluating internal validity are available: evaluating the potential for bias in a body of literature based on the study designs employed (levels of evidence), evaluating whether key study design features associated with the potential for bias were employed (quality assessment), and applying a judgement as to whether design elements of a study were likely to result in biased results given the specific context of the study (risk of bias assessment). The level of evidence framework for assessing internal validity assumes that internal validity can be determined based on the study design alone, and thus makes the strongest assumptions. Risk of bias assessments involve an evaluation of the potential for bias in the context of a specific study, and thus involve the least assumptions about internal validity. Quality assessment sits somewhere between the assumptions of these two. Because risk of bias assessment involves the least assumptions, this approach should be used to assess internal validity where possible. However, risk of bias instruments are not available for all study designs, some clinical questions may be addressed using multiple study designs, and some instruments that include an evaluation of internal validity also include additional components (e.g., evaluation of comprehensiveness of reporting, assessments of feasibility or an evaluation of external validity). Therefore, it may be necessary to embed questions related to risk of bias within existing quality assessment instruments. In this article, we overview the approaches to evaluating internal validity, highlight the current complexities, and propose ideas for approaching assessments of internal validity.