Evidence-based medicine has been hijacked: a report to David Sackett. J Clin Epidemiol. 2016;73:82-86. [PMID: 26934549 DOI: 10.1016/j.jclinepi.2016.02.012]

Disclosing conflict of interest does not mitigate healthcare bias and harm: It is time to sever industry ties

Disclosing conflict of interest is widely believed to mitigate bias and harm from industry entanglement in healthcare research, education and practice. However, evidence is lacking of disclosure-related mitigation. Radical and extensive severing of industry ties to health care is necessary to curtail industry-related bias and harm.

Practicing outcome-based medical care using pragmatic care trials

The current separation between medical research and care is an obstacle to essential aspects of good medical practice: the verification that care interventions actually deliver the good outcomes they promise, and the use of scientific methods to optimize care under uncertainty. Pragmatic care trials have been designed to address these problems. Care trials are all-inclusive randomized trials integrated into care. Every item of trial design is selected in the best medical interest of participating patients. Care trials can eventually show what constitutes good medical practice based on patient outcomes. In the meantime, care trials give clinicians and patients the scientific methods necessary for optimization of medical care when no one really knows what to do.We report the progress of 9 randomized care trials that were used to guide the endovascular or surgical management of 1212 patients with acute stroke, intracranial aneurysms, and arteriovenous malformations in a single center in an elective or acute care context. Care trials were used to address long-standing dilemmas regarding rival medical, surgical, or endovascular management options or to offer innovative instead of standard treatments. The trial methodology, by replacing unrepeatable treatment decisions by 1:1 randomized allocation whenever reliable knowledge was not available, had an immediate impact, transforming unverifiable dogmatic medical practice into verifiable outcome-based medical care. We believe the approach is applicable to all medical or surgical domains, but widespread adoption may require the revision of many currently prevalent views regarding the role of research in clinical practice.

Quality assessment of evidence-based clinical practice guidelines for the management of pregnant women with sickle cell disease using the AGREE II instrument: a systematic review

BACKGROUND

The management of pregnant women with sickle cell disease (SCD) poses a major challenge for maternal healthcare services owing to the potential for complications associated with morbidity and mortality. Trustworthy evidence-based clinical practice guidelines (CPGs) have a major impact on the positive outcomes of appropriate healthcare. The objective of this study was to critically appraise the quality of recent CPGs for SCD in pregnant women.

METHODS

Clinical questions were identified and the relevant CPG and bibliographic databases were searched and screened for eligible CPGs. Each CPG was appraised by four independent appraisers using the AGREE II Instrument. Inter-rater analysis was conducted.

RESULTS

Four eligible CPGs were appraised: American College of Obstetricians and Gynecologists (ACOG), National Heart, Lung, and Blood Institute (NHLBI), National Institute of Health and Care Excellence (NICE), and Royal College of Obstetricians and Gynaecologists (RCOG). Among them, the overall assessments of three CPGs (NICE, RCOG, NHLBI) scored greater than 70%; these findings were consistent with the high scores in the six domains of AGREE II, including:[1] scope and purpose,[2] stakeholder involvement,[3] rigor of development,[4] clarity of presentation,[5] applicability, and [6] editorial independence domains. Domain [3] scored (90%, 73%, 71%), domain [5] (90%, 46%, 47%), and domain [6] (71%, 77%, 52%) for NICE, RCOG, and NHLBI, respectively. Overall, the clinical recommendations were not significantly different between the included CPGs.

CONCLUSIONS

Three evidence-based CPGs presented superior methodological quality. NICE demonstrated the highest quality followed by RCOG and NHLBI and all three CPGs were recommended for use in practice.

Practice-based learning: an appropriate means to acquire the attitude and skills for evidence-based medicine

OBJECTIVES

  To evaluate a practice-based, self-directed EBM-course in an undergraduate medical curriculum in terms of EBM attitude and motivation beliefs.

METHODS

This study was conducted in a 4-week course of the first-year undergraduate medical curriculum, which takes place twice in an academic year. One group of students (n=210) received a normal EBM-module in November. A practice-based EBM-module was implemented in January for another group of students (n=130). We approached all students following the courses for participation in our research project. In a quasi-experimental design, a validated survey was used to assess students' EBM task value and self-efficacy on a 7-point Likert-scale. In the experimental group, complementary qualitative data were gathered on attitude and motivation by open evaluative questions.

RESULTS

Overall response rate was 93,5%, resulting in 191 students in the control group and 127 students in the experimental group. We did not find differences between the groups in terms of EBM task value and self-efficacy. However, the experimental group showed a higher increased perception of the importance of EBM in decision making in clinical practice (60.0% vs 77.2%; χ²(1, N=318) = 8.432, p=0.004). These students obtained a better understanding of the complexities and time-consuming nature of EBM in medical practice.

CONCLUSIONS

The practice-based EBM-course helps students to reflect on practice and knowledge critically. Our findings indicate that integrating clinical practice in the undergraduate learning environment fosters attitude and motivation, suggesting that practice-based learning in EBM education may advance student development as a critically reflective practitioner.

Evidence-based healthcare, knowledge translation, implementation science and radiography: What does it all mean?

Evidence-based healthcare is an approach to practice whereby decision making and service delivery considers the best available evidence, patient choice, local resources and clinical expertise. Although sound in theory, compliance with evidence-based practice is often far from optimal. To address this, related fields such as knowledge translation and implementation science have emerged, which largely focus on methods and techniques to facilitate the transfer of evidence into practice. This introductory article explains the key concepts of evidence-based healthcare by using the JBI Model as a framework to highlight the lifecycle of evidence-based information. Throughout the article we refer to exemplars from medical radiation to highlight these concepts.

A return to reasonableness and virtue in medical epistemology

The foundationalist and anti-foundationalist conceptions of medical knowledge have been at loggerheads for decades. Evidence-based medicine (EBM), the most prominent form of foundationalism, has attained wide appeal and acceptance among authorities. It proposes that evidence is the "base" upon which all clinical decisions should be grounded. Others have countered that the clinical encounter is far too complex for a singular base, and the different factors that impose on a clinical decision cannot be neatly and permanently ranked a priori. By its very nature, this anti-foundationalist outlook has resisted simplistic description, which is perhaps the reason it has not been as popular. In this paper, I provide a survey of the foundationalist and anti-foundationalist debate in medicine and defend anti-foundationalism on the basis that foundationalist approaches are anachronistic, and in the case of evidence-based medicine ultimately confuses inputs (evidence) for consideration in making a judgement with outputs (conclusions). I further propose that virtue ethics is inseparable from anti-foundationalism and conclude that the current infatuation with EBM implies something rather troubling; that physicians cannot be trusted to utilize their extensive training and skills to make reasonable decisions in the best interests of their patients. If this is in fact true, it suggests a crisis in virtue amongst medical professionals.

The surgeons and the COVID-19 pandemic

The current world Covid-19 pandemic has been the most discussed topic in the media and scientific journals. Fear, uncertainty, and lack of knowledge about the disease may be the significant factors that justify such reality. It has been known that the disease presents with a rapidly spreading, it is significantly more severe among the elderly, and it has a substantial global socioeconomic impact. Besides the challenges associated with the unknown, there are other factors, such as the deluge of information. In this regard, the high number of scientific publications, encompassing in vitro, case studies, observational and randomized clinical studies, and even systematic reviews add up to the uncertainty. Such a situation is even worse when considering that most healthcare professionals lack adequate knowledge to critically appraise the scientific method, something that has been previously addressed by some authors. Therefore, it is of utmost importance that expert societies supported by data provided by the World Health Organization and the National Health Department take the lead in spreading trustworthy and reliable information. The Brazilian College of Surgeons suggests in this document various initiatives that may help surgeons, healthcare providers, and patients who will have to face a surgical event under the pandemic.

Evidence-based medicine self-assessment, knowledge, and integration into daily practice: a survey among Romanian physicians and comparison between trainees and specialists

BACKGROUND

A gap between the attitude towards evidence-based medicine (EBM), knowledge and awareness has been reported among physicians from different parts of the world. However, no investigation on Romanian physicians is available in the scientific literature. Our study aimed, firstly, to assess EBM awareness and the knowledge used by Romanian physicians, and, secondly, to compare resident trainees with specialists.

METHODS

Romanian trainee and specialist physicians were invited to participate in this cross-sectional study. The study tool was an online questionnaire designed to explore their awareness, knowledge, usefulness, the attitude in medical documentation, and the use of professional EBM resources. Data were collected by Google Form from January 1st to April 30th, 2017, respecting the responders' anonymity. Two groups of physicians were investigated as trainees and specialists, respectively. Descriptive statistics (number, percentage, median and interquartile range) was used to describe the survey-related variables. Statistical significance on qualitative data was calculated with the Chi-square test, Fisher's exact test, or the Z-test for proportions.

RESULTS

Two hundred and 50 physicians participated in this study (68% trainees vs. 32% specialists). In both groups, a significantly high percentage was represented by women as compared to men (trainees 72.4%, specialists 70%). The correct definition of EBM was identified by most respondents (75.6%). Affirmatively, both trainees and specialists always looked at levels of evidence when reading scientific literature, but a small percentage (6.5% trainees and 3% specialists) adequately identified the uppermost types of evidence in the hierarchy. Almost a quarter of the respondents shared the name of mobile EBM resources that they used to support the daily practice. Only six out of the 49 listed mobile resources met the EBM criteria.

CONCLUSIONS

The participants proved to have limited knowledge of EBM and a positive attitude towards the concept. They made use of mobile medical resources without understanding which of these were evidence-based.

Timing of Renal Replacement Therapy for Severe Acute Kidney Injury in Critically Ill Patients

Acute kidney injury (AKI) affects many ICU patients and is responsible for increased morbidity and mortality. Although lifesaving in many situations, renal replacement therapy (RRT) may be associated with complications, and the appropriate timing of its initiation is still the subject of intense debate. An early initiation strategy can prevent some metabolic complications, whereas a delayed one may allow for renal function recovery in some patients without need for this costly and potentially dangerous technique. For years, most of the knowledge on this issue stemmed from observational studies or small randomized controlled trials. Recent randomized controlled trials have indicated that a watchful waiting strategy (in the absence of life-threatening conditions such as severe hyperkalemia or pulmonary edema) during severe AKI allowed many patients to escape RRT and did not seem to adversely affect survival compared with a strategy of immediate RRT. In addition, data suggest that a delayed strategy may reduce the rate of complications (such as catheter infection) and favor renal function recovery. Ongoing studies will have to both confirm these conclusions and clarify to what extent the delay in initiating RRT can be prolonged. Pending those results, the bulk of evidence suggests that, in the absence of potential severe complications of AKI, delaying RRT is a valid and safe strategy that may also allow for considerable cost savings.

E-Synthesis: A Bayesian Framework for Causal Assessment in Pharmacosurveillance

Background: Evidence suggesting adverse drug reactions often emerges unsystematically and unpredictably in form of anecdotal reports, case series and survey data. Safety trials and observational studies also provide crucial information regarding the (un-)safety of drugs. Hence, integrating multiple types of pharmacovigilance evidence is key to minimising the risks of harm.

Methods: In previous work, we began the development of a Bayesian framework for aggregating multiple types of evidence to assess the probability of a putative causal link between drugs and side effects. This framework arose out of a philosophical analysis of the Bradford Hill Guidelines. In this article, we expand the Bayesian framework and add “evidential modulators,” which bear on the assessment of the reliability of incoming study results. The overall framework for evidence synthesis, “E-Synthesis”, is then applied to a case study.

Results: Theoretically and computationally, E-Synthesis exploits coherence of partly or fully independent evidence converging towards the hypothesis of interest (or of conflicting evidence with respect to it), in order to update its posterior probability. With respect to other frameworks for evidence synthesis, our Bayesian model has the unique feature of grounding its inferential machinery on a consolidated theory of hypothesis confirmation (Bayesian epistemology), and in allowing any data from heterogeneous sources (cell-data, clinical trials, epidemiological studies), and methods (e.g., frequentist hypothesis testing, Bayesian adaptive trials, etc.) to be quantitatively integrated into the same inferential framework.

Conclusions: E-Synthesis is highly flexible concerning the allowed input, while at the same time relying on a consistent computational system, that is philosophically and statistically grounded. Furthermore, by introducing evidential modulators, and thereby breaking up the different dimensions of evidence (strength, relevance, reliability), E-Synthesis allows them to be explicitly tracked in updating causal hypotheses.

Development, Reporting, and Evaluation of Clinical Practice Guidelines

Clinical practice parameters have been published with greater frequency by professional societies and groups of experts. These publications run the gamut of practice standards, practice guidelines, consensus statements or practice advisories, position statements, and practice alerts. The definitions of these terms have been clarified in an accompanying article. In this article, we present the criteria for high-quality clinical practice parameters and outline a process for developing them, specifically the Delphi method, which is increasingly being used to build consensus among content experts and stakeholders. Several tools for grading the level of evidence and strength of recommendation are offered and compared. The speciousness of categorizing guidelines as evidence-based or consensus-based will be explained. We examine the recommended checklist for reporting and appraise the tools for evaluating a practice guideline. This article is geared toward developers and reviewers of clinical practice guidelines and consensus statements.

Clinical guidelines and patient related outcomes: summary of evidence and recommendations

Purpose

Good medical practice, evidence-based medicine (EBM) and clinical practice guidelines (CPG) have been recurring subjects in the scientific literature. EBM advocates argue that good medical practice should be guided by evidence-based CPG. On the other hand, critical authors of EBM methodology argue that various interests undermine the quality of evidence and reliability of CPG recommendations. The purpose of this paper is to evaluate patient related outcomes of CPG implementation, in light of EBM critics.

Design/methodology/approach

The authors opted for a rapid literature review.

Findings

There are few studies evaluating the effectiveness of CPG in patient-related outcomes. The systematic reviews found are not conclusive, although they suggest a positive impact of CPGs in relevant outcomes.

Research limitations/implications

This work was not a systematic review of literature, which is its main limitation. On the other hand, arguments from EBM and CPG critics were considered, and thus it can enlighten health institutions to recognize the caveats and to establish policies toward care improvement.

Originality/value

The paper is the first of its kind to discuss, based on the published literature, next steps toward better health practice, while acknowledging the caveats of this process.

Research paradigms and the politics of nursing knowledge: A reflective discussion

A standard view would suggest that research is a neutral apolitical activity. It neutralizes external pressures by its fidelity to robust scientific methods. However, politics is an inevitable part of human knowledge. Our knowledge of the world is always mediated by human priorities. What matters is therefore a contested and political debate rather a neutral accumulation of factual data. How researchers manage this varies. Research paradigms are one way in which research engages with knowledge. They frame knowledge within epistemological and ontological philosophies. In this paper, I will explore this view in relation to neo-positivism, qualitative research, Foucault and critical realism. I will argue that if nursing knowledge is to be effective it needs to acknowledge the political, particularly in the context of neoliberalism. Healthcare systems are having to cope with a social world increasingly dominated by market fundamentalism, extreme levels of inequality and a rise in xenophobia. These forces are undermining the provision of ethically sound health care, misdirecting research practice and contributing to a discourse of dehumanization. These forces need to be challenged politically and I will argue that epistemologically diverse approaches, alongside a realist ontology can provide a way forward for nursing research.

Appraisal of clinical practice guidelines for the management of attention deficit hyperactivity disorder (ADHD) using the AGREE II Instrument: A systematic review

BACKGROUND AND OBJECTIVE

High quality evidence-based clinical practice guidelines (CPGs) have a major impact on the appropriate diagnosis and management and positive outcomes. The evidence-based healthcare for patients with attention deficit hyperactive disorder (ADHD) is challenging. The objective of this study was to appraise the quality of published CPGs for ADHD.

METHODS

A systematic review was conducted for ADHD CPGs using CPG databases, DynaMed, PubMed, and Google Scholar. The quality of each included CPG was appraised by three independent appraisers using the Appraisal of Guidelines for Research & Evaluation II (AGREE II) instrument.

RESULTS

Six CPGs were critically reviewed. The AGREE II standardized domain scores revealed variation between the quality of these CPGs with the National Institute of Health and Care Excellence (NICE), University of Michigan Health System, and American Academy of Pediatrics CPGs as the top three. Overall, the recommendations for management of ADHD were similar in these CPGs.

CONCLUSIONS

Reporting of CPG development is often poorly documented. Guideline development groups should aim to follow the AGREE II criteria to improve the standards and quality of CPGs. The NICE CPG showed the best quality. Embedding the AGREE II appraisal of CPGs in the training and education of healthcare providers is recommended. The protocol for this study was published in PROSPERO (International prospective register of systematic reviews). Link: http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42017078712 and is additionally available from protocols.io. Link: https://dx.doi.org/10.17504/protocols.io.q27dyhn.

A Systems Approach to Understanding and Improving Research Integrity

Concern about the integrity of empirical research has arisen in recent years in the light of studies showing the vast majority of publications in academic journals report positive results, many of these results are false and cannot be replicated, and many positive results are the product of data dredging and the application of flexible data analysis practices coupled with selective reporting. While a number of potential solutions have been proposed, the effects of these are poorly understood and empirical evaluation of each would take many years. We propose that methods from the systems sciences be used to assess the effects, both positive and negative, of proposed solutions to the problem of declining research integrity such as study registration, Registered Reports, and open access to methods and data. In order to illustrate the potential application of systems science methods to the study of research integrity, we describe three broad types of models: one built on the characteristics of specific academic disciplines; one a diffusion of research norms model conceptualizing researchers as susceptible, "infected" and recovered; and one conceptualizing publications as a product produced by an industry comprised of academics who respond to incentives and disincentives.

Protect us from poor-quality medical research

Much of the published medical research is apparently flawed, cannot be replicated and/or has limited or no utility. This article presents an overview of the current landscape of biomedical research, identifies problems associated with common study designs and considers potential solutions. Randomized clinical trials, observational studies, systematic reviews and meta-analyses are discussed in terms of their inherent limitations and potential ways of improving their conduct, analysis and reporting. The current emphasis on statistical significance needs to be replaced by sound design, transparency and willingness to share data with a clear commitment towards improving the quality and utility of clinical research.

Research integrity and the law that never was

Evidence-based medicine is arguably among the most important innovations of the modern era, but publication bias and inadequate research transparency are serious issues affecting the very foundation of evidence-based practice. Despite this truth, these crucial issues have gone largely unaddressed or inadequately addressed for a distressingly long period of time. Regulatory efforts have thus far proven insufficient in eliminating these issues. Fortunately, the last 5 years in particular have seen developments that one hopes will contribute to the eradication of these issues and a future where we can look back on these issues as a sordid story of our past. However, much like the purported fixes of the past, time will be the final arbiter of the efficacy of remedial measures currently underway. This article chronicles the history of these issues, failed attempts to fix these issues, and what can be and is being done with the hope of bringing about true resolution.

Scientism, conflicts of interest, and the marginalization of ethics in medical education

AIM

This paper reports on the findings from 6 focus groups conducted with Australian medical students. The focus groups sought students' perspectives on how the influence of commercial interests on medical practice and education could be managed.

METHOD

We conducted 6 focus groups with medical students in New South Wales, Australia. Participants were recruited via student-run medical society and faculty e-mail lists. Forty-nine students from 6 medical schools in New South Wales participated. The research team reflected on the extent to which students uncritically appealed to science in the abstract as a management solution for conflicts of interest. Data analysis was largely inductive, looking for uses of scientific terminology, EBM, and appeals to "science" in the management of COI and applied theoretical analyses of scientism.

RESULTS

The students in our study suggested that science and evidence-based medicine, rather than ethics or professionalism, were the best tools to deal with undue influence and bias. This paper uses philosophy of science literature to critically examine these scientistic appeals to science and EBM as a means of managing the influence of pharmaceutical reps and commercial interests. We argue that a scientistic style of reasoning is reinforced through medical curricula and that students need to be made aware of the epistemological assumptions that underpin science, medicine, and EBM to address the ethical challenges associated with commercialised health care.

CONCLUSION

More work is needed to structure medical curricula to reflect the complexities of practice and realities of science. However, curricula change alone will not sufficiently address issues associated with commercial interests in medicine. For real change to occur, there needs to be a broader social and professional debate about the ways in which medicine and industry interact, and structural changes that restrict or mitigate commercial influences in educational, research, and policy settings.

From MARS to MAGIC: The remarkable journey through time and space of the Grading of Recommendations Assessment, Development and Evaluation initiative

For over 30 years, "evidence-based" clinical guidelines remained entrenched in an oversimplified, design-based, framework for rating the strength of evidence supporting clinical recommendations. The approach frequently equated the rating of evidence with that of the recommendations themselves. "Grading Recommendations Assessment, Development and Evaluation (GRADE)" has emerged as a proposed antidote to obsolete guideline methodology. GRADE sponsors and collaborators are in the process of attempting to amplify and extend the framework to encompass implementation and adaptation of guidelines, above and beyond the evaluation and rating of clinical research. Alternative schemes and models for such extensions are beginning to appear. This commentary reviews the strengths and weaknesses of GRADE with reference to other recent critiques. It considers the GRADE Working Group's "evidence-to-decision" extension of the evidence rating framework, together with proposed alternatives. It identifies pitfalls of the GRADE system's cooptation of relational processes necessary to the interpretation and uptake of recommendations that properly belong to end-users. It also identifies dangers inherent in blurring important boundaries between clinical and policy applications of guidelines. Finally, it addresses criticisms regarding the lack of a theoretical framework supporting the different facets of the GRADE approach and proposes a social constructivist orientation to clinical guideline development and use. Recommendations are offered to potential guideline developers and users regarding how to draw upon the strengths of the GRADE framework without succumbing to its pitfalls.

The scope and conventions of evidence-based medicine need to be widened to deal with "too much medicine"

In order that evidence-based medicine can prevent "too much medicine", it has to provide evidence in support of "gold standard" findings for use as diagnostic criteria, on which the assessment of other diagnostic tests and the outcomes of randomized controlled trials depend. When the results of such gold standard tests are numerical, cut-off points have to be positioned, also based on evidence, to identify those in whom offering a treatment can be justified. Such a diagnosis depends on eliminating conditions that mimic the one to be treated. The distributions of the candidate gold standard test results in those with and without the required outcome of treatment are then used with Bayes rule to create curves that show the probabilities of the outcome with and without treatment. It is these curves that are used to identify a cut-off point for offering a treatment to a patient and also to inform the patient's decision to accept or reject the suggested treatment. This decision is arrived at by balancing the probabilities of beneficial outcomes against the probabilities of harmful outcomes and other costs. The approach is illustrated with data from a randomized controlled trial on treating diabetic albuminuria with an angiotensin receptor blocker to prevent the development of the surrogate end-point of "biochemical nephropathy". The same approach can be applied to nonsurrogate outcomes such as death, disability, quality of life, relief of symptoms, and their prevention. Those with treatment-justifying diagnoses such as "diabetic albuminuria" usually form part of a broader group such as "type 2 diabetes mellitus". Any of these can be made the subject of evidence-based differential diagnostic strategies.

Valuing Healthcare Improvement: Implicit Norms, Explicit Normativity, and Human Agency

I argue that greater attention to human agency and normativity in both researching and practicing service improvement may be one strategy for enhancing improvement science, illustrating with examples from cancer screening. Improvement science tends to deliberately avoid explicit normativity, for paradigmatically coherent reasons. But there are good reasons to consider including explicit normativity in thinking about improvement. Values and moral judgements are central to social life, so an adequate account of social life must include these elements. And improvement itself is unavoidably normative: it assumes that things could and should be better than they are. I seek to show that normativity will always be implicated in the creation of evidence, the design of programs, the practice of healthcare, and in citizens' judgements about that care, and to make a case that engaging with this normativity is worthwhile.

The Emperor's New Clothes: a Critical Appraisal of Evidence-based Medicine

Evidence-Based Medicine (EBM) is the way we are expected to deliver our healthcare in the 21st century. It has been described as the integration of information from best available evidence with the doctor's experience and the patient's point of view. Unfortunately, the original meaning of EBM has been lost and the worldwide medical community has shifted the paradigm to Guidelines-Based Medicine, that has displaced the figures of the doctor and the patient from the decision-making process and relegated them to mere executor and final target of decisions taken by someone else. Problems related to the reliability of evidence and to the way guidelines are constructed, implemented and followed are discussed in detail. It is mandatory that the whole medical community takes responsibility and tries to reverse this apparently inexorable process so to re-establish a proper evidence-based care, where patients and their healing relation with practitioners are at the centre and where doctors are able to critically evaluate the available evidence and use it in light of their personal experience and knowledge.

Systematic overview and critical appraisal of meta-analyses of interventions in intensive care medicine

RATIONALE

Meta-analysed intervention effect estimates are perceived to represent the highest level of evidence. However, such effects and the randomized clinical trials which are included in them need critical appraisal before the effects can be trusted.

OBJECTIVE

Critical appraisal of a predefined set of all meta-analyses on interventions in intensive care medicine to assess their quality and assessed the risks of bias in those meta-analyses having the best quality.

METHODS

We conducted a systematic search to select all meta-analyses of randomized clinical trials on interventions used in intensive care medicine. Selected meta-analyses were critically appraised for basic scientific criteria, (1) presence of an available protocol, (2) report of a full search strategy, and (3) use of any bias risk assessment of included trials. All meta-analyses which qualified these criteria were scrutinized by full "Risk of Bias in Systematic Reviews" ROBIS evaluation of 4 domains of risks of bias, and a "Preferred Reporting Items for Systematic Reviews and Meta-Analyses" PRISMA evaluation.

RESULTS

We identified 467 meta-analyses. A total of 56 meta-analyses complied with these basic scientific criteria. We scrutinized the risks of bias in the 56 meta-analyses by full ROBIS evaluation and a PRISMA evaluation. Only 4 meta-analyses scored low risk of bias in all the 4 ROBIS domains and 41 meta-analyses reported all 27 items of the PRISMA checklist.

CONCLUSION

In contrast with what might be perceived as the highest level of evidence only 0.9% of all meta-analyses were judged to have overall low risk of bias.