Reporting quality in systematic reviews of in vitro studies: a systematic review

Critical appraisal tools used in systematic reviews of in vitro cell culture studies: A methodological study

Systematic reviews (SRs) of preclinical studies are marked with poor methodological quality. In vitro studies lack assessment tools to improve the quality of preclinical research. This methodological study aimed to identify, collect, and analyze SRs based on cell culture studies to highlight the current appraisal tools utilized to support the development of a validated critical appraisal tool for cell culture in vitro research. SRs, scoping reviews, and meta-analyses that included cell culture studies and used any type of critical appraisal tool were included. Electronic search, study selection, data collection and methodological quality (MQ) assessment tool were realized. Further, statistical analyses regarding possible associations and correlations between MQ and collected data were performed. After the screening process, 82 studies remained for subsequent analysis. A total of 32 different appraisal tools were identified. Approximately 60% of studies adopted pre-structured tools not designed for cell culture studies. The most frequent instruments were SYRCLE (n = 14), OHAT (n = 9), Cochrane Collaboration's tool (n = 7), GRADE (n = 6), CONSORT (n = 5), and ToxRTool (n = 5). The studies were divided into subgroups to perform statistical analyses. A significant association (OR = 5.00, 95% CI = 1.54-16.20, p = 0.008) was found between low MQ and chronic degenerative disorders as topic of SR. Several challenges in collecting information from the included studies led to some modifications related to the previously registered protocol. These results may serve as a basis for further development of a critical appraisal tool for cell culture studies capable of capturing all the essential factors related to preclinical research, therefore enhancing the practice of evidence-based.

RoBDEMAT: A risk of bias tool and guideline to support reporting of pre-clinical dental materials research and assessment of systematic reviews

OBJECTIVES

To develop a risk of bias tool for pre-clinical dental materials research studies that aims to support reporting of future investigations and improve assessment in systematic reviews.

METHODS

A four-stage process following EQUATOR network recommendations was followed, which included project launch, literature review, Delphi process and the tool finalization. With the support of the European Federation of Conservative Dentistry (EFCD) and the Dental Materials Group of the International Association for Dental Research (DMG-IADR), a total of 26 expert stakeholders were included in the development and Delphi vote of the initial proposal. The proposal was built using data gathered from the literature review stage. During this stage, recent systematic reviews featuring dental materials research, and risk of bias tools found in the literature were comprehensively scanned for bias sources. The experts thus reached a consensus for the items, domains and judgement related to the tool, allowing a detailed guide for each item and corresponding signalling questions.

RESULTS

The tool features nine items in total, spread between 4 domains, pertaining to the following types of bias: bias related to planning and allocation (D1), specimen preparation (D2), outcome assessment (D3) and data treatment and outcome reporting (D4). RoBDEMAT, as presented, features signalling questions and a guide that can be used for RoB judgement. Its use as a checklist is preferred over a final summary score.

CONCLUSION

RoBDEMAT is the first risk of bias tool for pre-clinical dental materials research, supported and developed by a broad group of expert stakeholders in the field, validating its future use.

CLINICAL SIGNIFICANCE

This new tool will contribute the study field by improving the scientific quality and rigour of dental materials research studies and their systematic reviews. Such studies are the foundation and support of future clinical research and evidence-based decisions.

Methodological assessment of systematic reviews of in-vitro dental studies

BACKGROUND

Systematic reviews of in-vitro studies, like any other study, can be of heterogeneous quality. The present study aimed to evaluate the methodological quality of systematic reviews of in-vitro dental studies.

METHODS

We searched for systematic reviews of in-vitro dental studies in PubMed, Web of Science, and Scopus databases published up to January 2022. We assessed the methodological quality of the systematic reviews using a modified "A MeaSurement Tool to Assess systematic Reviews" (AMSTAR-2) instrument. The 16 items, in the form of questions, were answered with yes, no, or py (partial yes). Univariable and multivariable linear regression models were used to examine the association between systematic review characteristics and AMSTAR-2 percent score. Overall confidence in the results of the systematic reviews was rated, based on weaknesses identified in critical and non-critical AMSTAR-2 items.

RESULTS

The search retrieved 908 potential documents, and after following the eligibility criteria, 185 systematic reviews were included. The most researched topics were ceramics and dental bonding. The overall rating for the confidence in the results was critically low in 126 (68%) systematic reviews. There was high variability in the response among the AMSTAR-2 items (0% to 75% positively answered). The univariable analyses indicated dental specialty (p = 0.03), number of authors (coef: 1.87, 95% CI: 0.26, 3.47, p = 0.02), and year of publication (coef: 2.64, 95% CI: 1.90, 3.38, p < 0.01) were significantly associated with the AMSTAR-2 percent score. Whereas, in the multivariable analysis only specialty (p = 0.01) and year of publication (coef: 2.60, 95% CI: 1.84, 3.35, p < 0.001) remained significant. Among specialties, endodontics achieved the highest AMSTAR-2 percent score.

CONCLUSIONS

The methods of systematic reviews of in vitro dental studies were suboptimal. Year of publication and dental specialty were associated with AMSTAR-2 scores. The overall rating of the confidence in the results was low and critically low for most systematic reviews.

Methodological quality assessment criteria for the evaluation of laboratory-based studies included in systematic reviews within the specialty of Endodontology: a development protocol

High quality systematic reviews in the field of Dentistry provide the most definitive overarching evidence for clinicians, guideline developers and healthcare policy makers to judge the foreseeable risks, anticipated benefits, and potential harms of dental treatment. In the process of carrying out a systematic review, it is essential that authors appraise the methodological quality of the primary studies they include, because studies which follow poor methodology will have a potentially serious negative impact on the overall strength of the evidence and the recommendations that can be drawn. In Endodontology, systematic reviews of laboratory studies have used quality assessment criteria developed subjectively by the individual authors as there are no comprehensive, well-structured, and universally accepted criteria that can be used objectively and universally to individual studies included in reviews. Unfortunately, these subjective criteria are likely to be inaccurately-defined, unreliably-applied, inadequately-analysed, unreasonably-biased, defective, and non-repeatable. The aim of the present paper is to outline the process to be followed in the development of comprehensive methodological quality assessment criteria to be used when evaluating laboratory studies that should be included in systematic reviews within Endodontology. The development of new methodological quality assessment criteria for appraising the laboratory-based studies included in systematic reviews within Endodontology will follow a three-stage process. First, a steering committee will be formed by the project leaders to develop a preliminary list of assessment criteria by modifying and adapting those already available, but with the addition of several new items relevant for Endodontology. The initial draft assessment criteria will be reviewed and refined by a Delphi Group (n=40) for their relevance and inclusion using a nine-point Likert scale. Second, the agreed items will then be discussed in an online or face-to-face meeting by a group of experts (n=10) to further refine the assessment criteria. Third, based on the feedback received from the online/face-to-face meeting, the steering committee will revise the quality assessment criteria and subsequently a group of authors will be selected to pilot the new system. Based on the feedback collected, the criteria may be revised further before being approved by the steering committee. The assessment criteria will be published in relevant journals, presented at national and international congresses/meetings, and will be freely available on a dedicated website. The steering committee will update the assessment criteria periodically based on feedback received from end-users.

Residual Adhesive Removal Methods for Rebonding of Debonded Orthodontic Metal Brackets: Systematic Review and Meta-Analysis

Debonding of orthodontic brackets is a common occurrence during orthodontic treatment. Therefore, the best option for treating debonded brackets should be indicated. This study aimed to evaluate the bond strength of rebonded brackets after different residual adhesive removal methods. This systematic review and meta-analysis was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. PubMed, Web of Science, The Cochrane Library, SciELO, Scopus, LILACS, IBECS, and BVS databases were screened up to December 2020. Bond strength comparisons were made considering the method used for removing the residual adhesive on the bracket base. A total of 12 studies were included for the meta-analysis. Four different adhesive removal methods were identified: sandblasting, laser, mechanical grinding, and direct flame. When compared with new orthodontic metallic brackets, bond strength of debonded brackets after air abrasion (p = 0.006), mechanical grinding (p = 0.007), and direct flame (p < 0.001) was significantly lower. The use of an erbium-doped yttrium aluminum garnet (Er:YAG) laser showed similar shear bond strength (SBS) values when compared with those of new orthodontic brackets (p = 0.71). The Er:YAG laser could be considered an optimal method for promoting the bond of debonded orthodontic brackets. Direct flame, mechanical grinding, or sandblasting are also suitable, obtaining clinically acceptable bond strength values.

The effect of the combined use of silver diamine fluoride and potassium iodide in disrupting the plaque biofilm microbiome and alleviating tooth discoloration: A systematic review

Silver diamine fluoride (SDF) is used in minimally invasive dentistry for arresting dental caries. However, discoloration of teeth is a significant side effect that has limited the use of SDF. Hence, the application of potassium iodide (KI) following SDF has been proposed to ameliorate the staining. Although antimicrobial activity is one of the major mechanisms of the caries-arresting effect of SDF, the antimicrobial potency of SDF/KI combination is unclear. Thus, the primary objective of this systematic review was to appraise the studies on the antimicrobial efficacy of SDF/KI combination on cariogenic microbes. The secondary objective was to summarize the evidence on the potential of KI in reducing the discoloration associated with the application of SDF. Electronic databases of Medline via PubMed, Cochrane Library, Web of Science, and EBSCO host were searched for English language manuscripts from January 2005 to 15th November 2020. The reference lists of these manuscripts were manually searched for additional studies. Twelve studies were included in the final analysis, seven of which have investigated the antimicrobial efficacy of SDF/KI, and the rest have examined the anti-staining potential of KI. The exploratory findings from the reviewed articles revealed the promising antimicrobial potential of SDF/KI on cariogenic microbes associated with dentine caries. There is, however, contradictory evidence on the effect of SDF/KI on tooth color. The reviewed in-vitro studies indicated significant effectiveness of KI in preventing staining. A clinical trial on primary dentition showed 25% reduction in the incidence of staining by SDF after applying KI, while a clinical study on root caries in adults showed no significant effect. Within the methodological limitations of this review, we conclude that for arresting dental caries, SDF could be combined with KI, as there may be a lower likelihood of staining. Further, well-designed clinical trials on the antimicrobial and anti-staining effect of SDF/KI are needed to obtain more robust evidence.

Quality assessment tools used in systematic reviews of in vitro studies: A systematic review

BACKGROUND

Systematic reviews (SRs) and meta-analyses (MAs) are commonly conducted to evaluate and summarize medical literature. This is especially useful in assessing in vitro studies for consistency. Our study aims to systematically review all available quality assessment (QA) tools employed on in vitro SRs/MAs.

METHOD

A search on four databases, including PubMed, Scopus, Virtual Health Library and Web of Science, was conducted from 2006 to 2020. The available SRs/MAs of in vitro studies were evaluated. DARE tool was applied to assess the risk of bias of included articles. Our protocol was developed and uploaded to ResearchGate in June 2016.

RESULTS

Our findings reported an increasing trend in publication of in vitro SRs/MAs from 2007 to 2020. Among the 244 included SRs/MAs, 126 articles (51.6%) had conducted the QA procedure. Overall, 51 QA tools were identified; 26 of them (51%) were developed by the authors specifically, whereas 25 (49%) were pre-constructed tools. SRs/MAs in dentistry frequently had their own QA tool developed by the authors, while SRs/MAs in other topics applied various QA tools. Many pre-structured tools in these in vitro SRs/MAs were modified from QA tools of in vivo or clinical trials, therefore, they had various criteria.

CONCLUSION

Many different QA tools currently exist in the literature; however, none cover all critical aspects of in vitro SRs/MAs. There is a need for a comprehensive guideline to ensure the quality of SR/MA due to their precise nature.

Protein-saliva interactions: a systematic review

Food industries are challenged to reformulate foods and beverages with higher protein contents to lower fat and sugar content. However, increasing protein concentration can reduce sensory acceptability due to astringency perception. Since the properties of food-saliva mixtures govern mouthfeel perception, understanding how saliva and protein interact is key to guide development of future protein-rich reformulations with optimal sensory attributes. Hence, this systematic review investigated protein-saliva interaction using both model and real human saliva, including a quality assessment. A literature search of five databases (Medline, Pubmed, Embase, Scopus and Web of Science) was undertaken covering the last 20 years, yielding 36 604 articles. Using pre-defined criteria, this was reduced to a set of 33 articles with bulk protein solutions (n = 17), protein-stabilized emulsions (n = 13) and protein-rich food systems (n = 4). Interaction of dairy proteins, lysozyme and gelatine with model or human saliva dominated the literature. The pH was shown to have a strong effect on electrostatic interaction of proteins with negatively-charged salivary mucins, with greater interactions occurring below the isoelectric point of proteins. The effect of protein concentration was unclear due to the limited range of concentrations being studied. Most studies employed a 1 : 1 w/w protein : saliva ratio, which is not representative of true oral conditions. The interaction between protein and saliva appears to affect mouthfeel through aggregation and increased friction. The searches identified a gap in research on plant proteins. Accurate simulation of in vivo oral conditions should clarify understanding of protein-saliva interaction and its influence on sensory perception.

Comparison of freshly cultured versus freshly thawed (cryopreserved) mesenchymal stem cells in preclinical in vivo models of inflammation: a protocol for a preclinical systematic review and meta-analysis

BACKGROUND

Mesenchymal stem cells (MSCs) are multipotent cells that demonstrate therapeutic potential for the treatment of acute and chronic inflammatory-mediated conditions. Especially for acute conditions, it is critical to have a readily available freshly thawed (cryopreserved) MSC product for rapid administration. Although controversial, some studies suggest that MSCs may lose their functionality with cryopreservation which in turn could render them non-efficacious.

OBJECTIVE

In controlled preclinical in vivo models of inflammation, to determine if there are differences in surrogate measures of preclinical efficacy between freshly cultured and freshly thawed MSCs METHODS/DESIGN: A systematic search for pre-clinical in vivo inflammatory model studies will compare freshly cultured to freshly thawed MSCs from any source. The primary outcomes will include measures of in vivo preclinical efficacy; secondary outcomes will include measures of in vitro MSC potency. Electronic searches for MEDLINE and EMBASE will be constructed and reviewed by the Peer Review of Electronic Search Strategies (PRESS) process. If applicable, study outcomes will be meta-analyzed using a random effects model. Risk of bias will be assessed by the SYRCLE "Risk of Bias" assessment tool for preclinical in vivo studies.

DISCUSSION

The results of this systematic review will provide translational scientists, clinical trialists, health regulators, and the clinical and public community with the current pre-clinical evidence base related to the efficacy and potency of freshly cultured versus freshly thawed MSCs, help identify evidence gaps, and guide future related research.

SYSTEMATIC REVIEW REGISTRATION

Protocol is submitted to PROSPERO for registration (pending confirmation) and will be submitted to Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies (CAMARADES) for public posting.

A Systematic Approach to Review of in vitro Methods in Brain Tumour Research (SAToRI-BTR): Development of a Preliminary Checklist for Evaluating Quality and Human Relevance

Background

A wide range of human in vitro methods have been developed and there is considerable interest in the potential of these studies to address questions related to clinical (human) use of drugs, and the pathobiology of tumours. This requires agreement on how to assess the strength of evidence available (i.e., quality and quantity) and the human-relevance of such studies. The SAToRI-BTR (Systematic Approach To Review of in vitro methods in Brain Tumour Research) project seeks to identify relevant appraisal criteria to aid planning and/or evaluation of brain tumour studies using in vitro methods.

Objectives

To identify criteria for evaluation of quality and human relevance of in vitro brain tumour studies; to assess the general acceptability of such criteria to senior scientists working within the field.

Methods

Stage one involved identification of potential criteria for evaluation of in vitro studies through: (1) an international survey of brain tumour researchers; (2) interviews with scientists, clinicians, regulators, and journal editors; (3) analysis of relevant reports, documents, and published studies. Through content analysis of findings, an initial list of criteria for quality appraisal of in vitro studies of brain tumours was developed. Stage two involved review of the criteria by an expert panel (Delphi process).

Results

Results of stage one indicated that methods for and quality of review of in vitro studies are highly variable, and that improved reporting standards are needed. 129 preliminary criteria were identified; duplicate and highly context-specific items were removed, resulting in 48 criteria for review by the expert (Delphi) panel. 37 criteria reached agreement, resulting in a provisional checklist for appraisal of in vitro studies in brain tumour research.

Conclusion

Through a systematic process of collating assessment criteria and subjecting these to expert review, SAToRI-BTR has resulted in preliminary guidance for appraisal of in vitro brain tumour studies. Further development of this guidance, including investigating strategies for adaptation and dissemination across different sub-fields of brain tumour research, as well as the wider in vitro field, is planned.

The collaboration and reporting quality of social welfare systematic reviews in the Campbell Collaboration online library

Background

To analyze the collaboration and reporting quality of the systematic reviews of social welfare in the Campbell collaboration online library.

Methods

The Campbell collaboration online library was searched for systematic reviews of social welfare and the basic information extracted in order to assess the reporting quality of systematic reviews using a MOOSE checklist. BICOMS-2 and UCINET software were used to produce the social network, and Comprehensive Meta Analysis (Version 2) and STATA 13.0 were used to analyze the related data.

Results

Fifty-seven systematic reviews of social welfare were included. Twenty-eight items of the included social welfare systematic reviews were rated as complete (≥70%). There were significant differences between ≤2013 and ≥ 2014 in five items. These differences were as follows: research published by one organization or more than one organization in one item, more than three authors or less than four authors in two items, and one country or more than one country in six items. It’s completed about researches with more than one organization, three authors or more than one country. Some items were found to have a low reporting rate of studies published before 2014, by one organization, with less than four authors or one country, respectively. The social network of authors and organizations showed good collaboration.

Conclusions

Some items could be further improved with regard to the rate of reporting systematic reviews of social welfare in the Campbell collaboration online library. This could improve the overall quality of social welfare systematic reviews.