Bias Assessment in Outcomes Research: The Role of Relative Versus Absolute Approaches

The efficacy and safety of venous thromboembolism prophylaxis among hospitalized cirrhotic patients: Systematic review and meta-analysis

BACKGROUND AND STUDY AIMS

This meta-analysis aims to evaluate the effect of prophylactic anticoagulation on incidence of venous thromboembolism (VTE), bleeding events, and mortality in hospitalized cirrhotic patients.

METHODS

We utilized the following databases (EMBASE, PubMed, MedRxiv and google-scholar) to search for studies that satisfied the reviewers pre-specified inclusion criteria. The incidence of VTE, bleeding risks, and mortality were assessed using a quality effect meta-analytic model.

RESULTS

From screening of 539 studies, a total of 9 studies (n = 6275 patients) satisfied inclusion criteria. Our results suggested no significant difference in the primary outcome of VTE events in both groups of cirrhotic patients who received and did not receive anticoagulation for VTE prophylaxis, (OR:0.9, 95 % CI:0.50-1.62). Similarly, there was a non-significant reduced risk of bleeding events in hospitalized cirrhotic amongst patient cohorts receiving VTE prophylaxis compared to those who did not (OR:0.51, 95 % CI:0.22-1.14). Analysis of three studies showed no significant difference in mortality in both groups (OR 1.02, 95 % CI:0.8-1.31).

CONCLUSION

In a pooled examination of studies evaluating outcomes in patients exposed to VTE prophylactic anticoagulation, we found no significant difference in the burden of VTE or mortality between treated and untreated patients with chronic liver disease (CLD). The retrospective design of a plurality of studies enrolled in the review meant further prospective studies are needed to objectively ascertain the efficacy and safety of VTE prophylaxis amongst patient cohorts with CLD.

The Clinicopathological and Prognostic Value of CCR7 Expression in Breast Cancer Throughout the Literature: A Systematic Review and Meta-Analysis

Background/Objective: This study aimed to determine the clinicopathological findings and prognostic value of chemokine receptor 7 (CCR7) expression in patients with breast cancer (BC). Methods: Up to the 25th of March 2025, a search was conducted using five databases: PubMed, Embase, Scopus, Medline, and Web of Science. The methodological standards for the epidemiological research scale were used to assess the quality of the included articles, and Stata software (Stata 19) was used to synthesize the meta-analysis. Results: We considered 12 of 853 studies that included 3119 patients with BC. High CCR7 expression was not associated with age (odds ratio [OR] 0.82, 95% confidence interval [CI] 0.66-1.03); clinicopathological findings, including tumor size (OR 1.062, 95% CI 0.630-1.791); clinical stage (OR 1.753, 95% CI 0.231-13.304); nodal metastasis (OR 1.252, 95% CI 0.571-2.741); or histological differentiation (OR 1.167, 95% CI 0.939-1.450). CCR7 expression did not affect overall survival (hazard ratio 0.996, 95% CI 0.659-1.505). Conclusions: Our quantitative analysis did not reveal an association between CCR7 expression and poor clinicopathological or prognostic features in BC patients. Because of the high heterogeneity and potential publication bias, large high-quality studies are required to further confirm these findings.

Mastering meta-analysis in Microsoft Excel with MetaXL add-in: A comprehensive tutorial and guide to meta-analysis

RATIONALE

Meta-analysis, a powerful technique for combining effect estimates from multiple studies, enhances statistical power and precision. However, its adoption can be hindered by challenges in statistical interpretation and the complexity of specialized software. MetaXL, a freely available Microsoft Excel add-in, aims to mitigate these barriers by providing comprehensive support and facilitating seamless integration of meta-analytical results into research publications.

AIMS AND OBJECTIVES

This tutorial illustrates the practical application of MetaXL for synthesizing meta-analytical evidence, with a focus on common effect sizes and their presentation.

METHOD

This paper reintroduce MetaXL's functions and provide concise explanations of common effect sizes employed in meta-analysis. The tutorial delves into fundamental concepts such as the selection of appropriate effect sizes for pooling and the choice of meta-analytical models. Eight illustrative examples are presented, incorporating diverse effect sizes and data formats, including scenarios involving incidence rate ratios, weighted and standardized mean differences, hazard ratios, and prevalence. Additionally, key concepts in network meta-analysis are discussed, along with their implementation in MetaXL. MetaXL provides convenient access to data formatting templates tailored to various data types and effect sizes encountered in included studies.

RESULTS AND CONCLUSION

This tutorial offers researchers, particularly those with limited resources, detailed explanations and insights into commonly used methodologies for pooling effect sizes. Furthermore, it introduces the new Excel functions that comes with the MetaXL add-in. Accurate population of this function and adherence to the correct format are essential to ensure error-free analyzes.

The revised JBI critical appraisal tool for the assessment of risk of bias for cohort studies

Cohort studies are a robust analytical observational study design that explore the difference in outcomes between two cohorts, differentiated by their exposure status. Despite being observational in nature, they are often included in systematic reviews of effectiveness, particularly when randomized controlled trials are limited or not feasible. Like all studies included in a systematic review, cohort studies must undergo a critical appraisal process to assess the extent to which a study has considered potential bias in its design, conduct, or analysis. Critical appraisal tools facilitate this evaluation. This paper introduces the revised critical appraisal tool for cohort studies, completed by the JBI Effectiveness Methodology Group, who are currently revising the suite of JBI critical appraisal tools for quantitative study designs. The revised tool responds to updates in methodological guidance from the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group and reporting guidance from PRISMA 2020, providing a robust framework for evaluating risk of bias in a cohort study. Transparent and rigorous assessment using this tool will assist reviewers in understanding the validity and relevance of the results and conclusions drawn from a systematic review that includes cohort studies. This may contribute to better evidence-based decision-making in health care. This paper discusses the key changes made to the tool, outlines justifications for these changes, and provides practical guidance on how this tool should be interpreted and applied by systematic reviewers.

Comparison of postoperative pain in children after maintenance anaesthesia with propofol or sevoflurane: a systematic review and meta-analysis

BACKGROUND

Propofol and sevoflurane are two of the most commonly used anaesthetics for paediatric surgery. Data from some clinical trials suggest that postoperative pain incidence is lower when propofol is used for maintenance of anaesthesia compared with sevoflurane, although this is not clear.

METHODS

This meta-analysis compared postoperative pain following maintenance of anaesthesia with propofol or sevoflurane in paediatric surgeries. PubMed Medline, Embase, Scopus, Web of Science and Cochrane Library were searched for randomised controlled trials (RCTs) that compared postoperative pain between sevoflurane and propofol anaesthesia in children. After quality assessment, a meta-analysis was carried out using bias-adjusted inverse heterogeneity methods, heterogeneity using I2 and publication bias using Doi plots.

RESULTS

In total, 13 RCTs with 1174 children were included. The overall synthesis suggested nearly two-fold higher odds of overall postoperative pain in the sevoflurane group compared with the propofol group (odds ratio [OR] 1.88, 95% confidence interval [CI] 1.12-3.15, I2=58.2%). Further, children in the sevoflurane group had higher odds of having higher pain scores (OR 3.18, 95% CI 1.83-5.53, I2=20.9%), and a 60% increase in the odds of requiring postoperative rescue analgesia compared with propofol (OR 1.60, 95% CI 0.89-2.88, I2=58.2%).

CONCLUSIONS

Children maintained on inhalational sevoflurane had higher odds of postoperative pain compared with those maintained on propofol. The results also suggest that sevoflurane is associated with higher odds of needing postoperative rescue analgesia compared with propofol.

REGISTRATION

The protocol for this systematic review and meta-analysis was registered on the International Prospective Register of Systematic Reviews (PROSPERO) with registration ID CRD42023445913.

Tranexamic Acid for Traumatic Injury in the Emergency Setting: A Systematic Review and Bias-Adjusted Meta-Analysis of Randomized Controlled Trials

STUDY OBJECTIVE

Traumatic injury causes a significant number of deaths due to bleeding. Tranexamic acid (TXA), an antifibrinolytic agent, can reduce bleeding in traumatic injuries and potentially enhance outcomes. Previous reviews suggested potential TXA benefits but did not consider the latest trials.

METHODS

A systematic review and bias-adjusted meta-analysis were performed to assess TXA's effectiveness in emergency traumatic injury settings by pooling estimates from randomized controlled trials. Researchers searched Medline, Embase, and Cochrane Central for randomized controlled trials comparing TXA's effects to a placebo in emergency trauma cases. The primary endpoint was 1-month mortality. The methodological quality of the trials underwent assessment using the MASTER scale, and the meta-analysis applied the quality-effects method to adjust for methodological quality.

RESULTS

Seven randomized controlled trials met the set criteria. This meta-analysis indicated an 11% decrease in the death risk at 1 month after TXA use (odds ratio [OR] 0.89, 95% confidence interval [CI] 0.84 to 0.95) with a number needed to treat of 61 to avoid 1 additional death. The meta-analysis also revealed reduced 24-hour mortality (OR 0.76, 95% CI 0.65 to 0.88) for TXA. No compelling evidence of increased vascular occlusive events emerged (OR 0.96, 95% CI 0.73 to 1.27). Subgroup analyses highlighted TXA's effectiveness in general trauma versus traumatic brain injury and survival advantages when administered out-of-hospital versus inhospital.

CONCLUSIONS

This synthesis demonstrates that TXA use for trauma in emergencies leads to a reduction in 1-month mortality, with no significant evidence of problematic vascular occlusive events. Administering TXA in the out-of-hospital setting is associated with reduced mortality compared to inhospital administration, and less mortality with TXA in systemic trauma is noted compared with traumatic brain injury specifically.

The revised JBI critical appraisal tool for the assessment of risk of bias for quasi-experimental studies

Systematic reviews of effectiveness offer a rigorous synthesis of the best evidence available regarding the effects of interventions or treatments. Randomized controlled trials are considered the optimal study design for evaluating the effectiveness of interventions and are the ideal study design for inclusion in a systematic review of effectiveness. In the absence of randomized controlled trials, quasi-experimental studies may be relied on to provide information on treatment or intervention effectiveness. However, such studies are subject to unique considerations regarding their internal validity and, consequently, the assessment of the risk of bias of these studies needs to consider these features of design and conduct. The JBI Effectiveness Methodology Group has recently commenced updating the suite of JBI critical appraisal tools for quantitative study designs to align with the latest advancements in risk of bias assessment. This paper presents the revised critical appraisal tool for risk of bias assessment of quasi-experimental studies; offers practical guidance for its use; provides examples for interpreting the results of risk of bias assessment; and discusses major changes from the previous version, along with the justifications for those changes.

Comparison of bias adjustment in meta-analysis using data-based and opinion-based methods

INTRODUCTION

Several methods exist for bias adjustment of meta-analysis results, but there has been no comprehensive comparison with unadjusted methods. We compare 6 bias-adjustment methods with 2 unadjusted methods to examine how these different methods perform.

METHODS

We re-analyzed a meta-analysis that included 10 randomized controlled trials. Two data-based methods (Welton's data-based approach and Doi's quality effects model) and 4 opinion-informed methods (opinion-based approach, opinion-based distributions combined statistically with data-based distributions, numerical opinions informed by data-based distributions, and opinions obtained by selecting areas from data-based distributions) were used to incorporate methodological quality information into the meta-analytical estimates. The results of these 6 methods were compared with 2 unadjusted models: the DerSimonian-Laird random effects model and Doi's inverse variance heterogeneity model.

RESULTS

The 4 opinion-based methods returned the random effects model estimates with wider uncertainty. The data-based and quality effects methods returned different results and aligned with the inverse variance heterogeneity method with some minor downward bias adjustment.

CONCLUSION

Opinion-based methods seem to only add uncertainty rather than bias adjust.

Common tool structures and approaches to risk of bias assessment: implications for systematic reviewers

There are numerous tools available to assess the risk of bias in individual studies in a systematic review. These tools have different structures, including scales and checklists, which may or may not separate their items by domains. There are also various approaches and guides for the process, scoring, and interpretation of risk of bias assessments, such as value judgments, quality scores, and relative ranks. The objective of this commentary, which is part of the JBI Series on Risk of Bias, is to discuss some of the distinctions among different tool structures and approaches to risk of bias assessment and the implications of these approaches for systematic reviewers.

Revising the JBI quantitative critical appraisal tools to improve their applicability: an overview of methods and the development process

JBI offers a suite of critical appraisal instruments that are freely available to systematic reviewers and researchers investigating the methodological limitations of primary research studies. The JBI instruments are designed to be study-specific and are presented as questions in a checklist. The JBI instruments have existed in a checklist-style format for approximately 20 years; however, as the field of research synthesis expands, many of the tools offered by JBI have become outdated. The JBI critical appraisal tools for quantitative studies (eg, randomized controlled trials, quasi-experimental studies) must be updated to reflect the current methodologies in this field. Cognizant of this and the recent developments in risk-of-bias science, the JBI Effectiveness Methodology Group was tasked with updating the current quantitative critical appraisal instruments. This paper details the methods and rationale that the JBI Effectiveness Methodology Group followed when updating the JBI critical appraisal instruments for quantitative study designs. We detail the key changes made to the tools and highlight how these changes reflect current methodological developments in this field.

Methodological quality assessment should move beyond design specificity

OBJECTIVE

This study aimed to assess the utility of a unified tool (MASTER) for bias assessment against design-specific tools in terms of content and coverage.

METHODS

Each of the safeguards in the design-specific tools was compared and matched to safeguards in the unified MASTER scale. The design-specific tools were the JBI, Scottish Intercollegiate Guidelines Network (SIGN), and the Newcastle-Ottawa Scale (NOS) tools for analytic study designs. Duplicates, safeguards that could not be mapped to the MASTER scale, and items not applicable as safeguards against bias were flagged and described.

RESULTS

Many safeguards across the JBI, SIGN, and NOS tools were common, with a minimum of 10 to a maximum of 23 unique safeguards across various tools. These 3 design-specific toolsets were missing 14 to 26 safeguards from the MASTER scale. The MASTER scale had complete coverage of safeguards within the 3 toolsets for analytic designs.

CONCLUSIONS

The MASTER scale provides a unified framework for bias assessment of analytic study designs, has good coverage, avoids duplication, has less redundancy, and is more convenient when used for methodological quality assessment in evidence synthesis. It also allows assessment across designs that cannot be done using a design-specific tool.

Assessing the risk of bias of quantitative analytical studies: introducing the vision for critical appraisal within JBI systematic reviews

A key step in the systematic review process is the assessment of the methodological quality (or risk of bias) of the included studies. At JBI, we have developed several tools to assist with this evaluation. As evidence synthesis methods continue to evolve, it has been necessary to revise and reflect on JBI's current approach to critical appraisal and to plan a strategy for the future. In this first paper of a series focusing on risk of bias assessment, we introduce our vision for risk of bias assessment for JBI. In future papers in this series, the methodological approach taken for this revision process will be discussed, along with the revised tools and guidance for using these tools.

The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials

JBI recently began the process of updating and revising its suite of critical appraisal tools to ensure that these tools remain compatible with recent developments within risk of bias science. Following a rigorous development process led by the JBI Effectiveness Methodology Group, this paper presents the revised critical appraisal tool for the assessment of risk of bias for randomized controlled trials. This paper also presents practical guidance on how the questions of this tool are to be interpreted and applied by systematic reviewers, while providing topical examples. We also discuss the major changes made to this tool compared to the previous version and justification for why these changes facilitate best-practice methodologies in this field.

Clinicopathological and prognostic value of chemokine receptor CCR7 expression in head and neck squamous cell carcinoma: a systematic review and meta-analysis

OBJECTIVES

This study aimed to identify the clinicopathological characteristics and prognostic value of CC chemokine receptor 7 (CCR7) expression in patients with head and neck squamous cell carcinoma (HNSSC).

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were followed in this meta-analysis. Up to the 2^nd of July 2022 a search was conducted using five databases: PubMed, Embase, Scopus, ProQuest, and Web of Science. The methodological standards for the epidemiological research scale were used to assess the quality of the included articles, and Stata software was used to synthesise the meta-analysis.

RESULTS

We considered 13 of the 615 studies which included 1005 HNSCC patients. High expression of CCR7 increased the pooled odds ratio (OR) of advanced stage, tumour size, metastasis and recurrence by 2.82 [95% confidence interval (CI) 1.84 to 4.33], 2.48 (95% CI 1.68, to 3.67), 3.57, 95% CI 2.25 to 5.05) and 3.93 (95% CI 2.03 to 7.64), respectively. High CCR7 reduced overall patient survival [hazard ratio 2.62 (95% CI 1.59 to 4.32)].

CONCLUSION

This study showed that high expression of CCR7 in HNSCC tumours was significantly associated with worse clinicopathological and survival outcomes, suggesting that CCR7 and its pathway could be potential therapeutic strategies for HNSCC.

Accuracy of artificial intelligence-assisted detection of Oral Squamous Cell Carcinoma: A systematic review and meta-analysis

Oral Squamous Cell Carcinoma (OSCC) is an aggressive tumor with a poor prognosis. Accurate and timely diagnosis is therefore essential for reducing the burden of advanced disease and improving outcomes. In this meta-analysis, we evaluated the accuracy of artificial intelligence (AI)-assisted technologies in detecting OSCC. We included studies that validated any diagnostic modality that used AI to detect OSCC. A search was performed in six databases: PubMed, Embase, Scopus, Cochrane Library, ProQuest, and Web of Science up to 15 Mar 2022. The Quality Assessment Tool for Diagnostic Accuracy Studies was used to evaluate the included studies' quality, while the Split Component Synthesis method was utilized to quantitatively synthesize the pooled diagnostic efficacy estimates. We considered 16 out of the 566 yielded studies, which included twelve different AI models with a total of 6606 samples. The summary sensitivity, summary specificity, positive and negative likelihood ratios as well as the pooled diagnostic odds ratio were 92.0 % (95 % confidence interval [CI] 86.7-95.4 %), 91.9 % (95 % CI 86.5-95.3 %), 11.4 (95 % CI 6.74-19.2), 0.087 (95 % CI 0.051-0.146) and 132 (95 % CI 62.6-277), respectively. Our findings support the capability of AI-assisted systems to detect OSCC with high accuracy, potentially aiding the histopathological examination in early diagnosis, yet more prospective studies are needed to justify their use in the real population.

Diagnostic accuracy of multiplex nucleic acid amplification tests for Campylobacter infection: a systematic review and meta-analysis

Campylobacter infection is one of the most frequently reported foodborne diseases with approximately 230,000 and 1.5 million cases each year in Europe and the USA, respectively. Culture methods are the reference for the diagnosis of Campylobacter infections; however, these methods are complex and time-consuming. Multiplex nucleic acid amplification test is favored due to its rapidity, automatization in the procedure followed and the quick simultaneous testing of numerous foodborne pathogens. The aim of this meta-analysis was to evaluate the accuracy of these tests for the diagnosis of Campylobacter infection. Scopus, Science Direct, PubMed, Web of Science, and Mendeley were searched for peer-reviewed articles. The split component synthesis method with the use of the inverse variance heterogeneity model was chosen for the quantitative meta-analysis. Sensitivity analysis was performed by age category and index test. The literature search found 34 studies involving 28,105 patients with suspected gastroenteritis. The sensitivity and specificity were 95.3% (92.3; 97.1) and 97.1% (95.1; 98.3), respectively, and AUC (area under the curve) was 0.963 (0.947; 0.974). Pediatric patients had a lower sensitivity (87.4, 48.2; 98.1) and higher specificity (99.2, 91.6; 99.9) estimate compared to all ages category (sensitivity 95.3, 91.3; 97.5, specificity 96.7, 93.7; 98.3). Among the various index tests, Seeplex/Allplex and Amplidiag/Novodiag had the lowest estimate for sensitivity (88.9, 73.8; 95.8) and specificity (95.2, 86; 98.4), respectively. BDMax had the highest (sensitivity 98.1, 96.1; 99 and specificity 98.5, 97; 99.3). Multiplex nucleic acid tests showed excellent accuracy and could play an influential role in diagnosing Campylobacter infections.