Reevaluation of statistically significant meta-analyses in advanced cancer patients using the Hartung-Knapp method and prediction intervals - a methodological study

Avoiding common mistakes in meta-analysis: Understanding the distinct roles of Q, I-squared, tau-squared, and the prediction interval in reporting heterogeneity

In any meta-analysis, it is critically important to report the dispersion in effects as well as the mean effect. If an intervention has a moderate clinical impact on average we also need to know if the impact is moderate for all relevant populations, or if it varies from trivial in some to major in others. Or indeed, if the intervention is beneficial in some cases but harmful in others. Researchers typically report a series of statistics such as the Q-value, the p-value, and I2 , which are intended to address this issue. Often, they use these statistics to classify the heterogeneity as being low, moderate, or high and then use these classifications when considering the potential utility of the intervention. While this practice is ubiquitous, it is nevertheless incorrect. The statistics mentioned above do not actually tell us how much the effect size varies. Classifications of heterogeneity based on these statistics are uninformative at best, and often misleading. My goal in this paper is to explain what these statistics do tell us, and that none of them tells us how much the effect size varies. Then I will introduce the prediction interval, the statistic that does tell us how much the effect size varies, and that addresses the question we have in mind when we ask about heterogeneity. This paper is adapted from a chapter in "Common Mistakes in Meta-Analysis and How to Avoid Them." A free PDF of the book is available at https://www.Meta-Analysis.com/rsm.

Endovascular thrombectomy after acute ischemic stroke of the basilar artery: a meta-analysis of four randomized controlled trials

BACKGROUND

Previous randomized controlled trials (RCTs) and meta-analyses were underpowered to demonstrate the superiority of endovascular thrombectomy (EVT) over medical therapy (MEDT) in the treatment of acute ischemic stroke due to large vessel occlusion of the posterior circulation (PC-LVO). We performed an updated systematic review and meta-analysis after the publication of the BAOCHE and ATTENTION trials to determine whether EVT can benefit patients presenting with PC-LVO.

METHODS

Using Nested Knowledge, we screened literature for RCTs on EVT in PC-LVO. The primary outcome was 90-day modified Rankin Scale (mRS) score 0-3, and secondary outcomes included 90-day mRS score 0-2, 90-day mortality, and rate of symptomatic intracranial hemorrhage (sICH). A random-effects model was used to compute rate ratios (RRs) and their corresponding 95% confidence intervals (CIs).

RESULTS

Four RCTs with 988 patients, 556 patients in the EVT arm and 432 patients in the MEDT arm, were included in the meta-analysis. EVT resulted in significantly higher rates of mRS score 0-3 (RR=1.54; 95% CI 1.16 to 2.04; P=0.002) and functional independence (RR=1.83; 95% CI 1.08 to 3.08; P=0.024), and lower rates of mortality (RR=0.76; 95% CI 0.65 to 0.90; P=0.002) at 90-day follow-up compared with MEDT alone. However, EVT patients had higher rates of sICH (RR=7.48; 95% CI 2.27 to 24.61; P<0.001).

CONCLUSIONS

EVT conferred significant patient benefit over MEDT alone in the treatment of PC-LVO. Future studies should better define patients for whom EVT is futile and determine factors that contribute to higher rates of sICH.

Evaluation of 'implications for research' statements in systematic reviews of interventions in advanced cancer patients - a meta-research study

BACKGROUND

Implications for research (IfR) sections are an important part of systematic reviews (SRs) to inform health care researchers and policy makers. PRISMA 2020 recommends reporting IfR, while Cochrane Reviews require a separate chapter on IfR. However, it is unclear to what extent SRs discuss IfR. We aimed i) to assess whether SRs include an IfR statement and ii) to evaluate which elements informed IfR statements.

METHODS

We conducted a meta-research study based on SRs of interventions in advanced cancer patients from a previous project (CRD42019134904). As suggested in the Cochrane Handbook, we assessed if the following predefined variables were referred to in IfR statements: patient, intervention, control, outcome (PICO) and study design; concepts underlying Grading of Recommendations, Assessment, Development and Evaluation (GRADE) domains: risk of bias, inconsistency, indirectness, imprecision, publication bias. Data were independently extracted by three reviewers after piloting the data extraction form. Discrepancies were resolved in weekly in-depth discussions.

RESULTS

We included 261 SRs. The majority evaluated a pharmacological intervention (n = 244, 93.5%); twenty-nine were Cochrane Reviews (11.1%). Four out of five SRs included an IfR statement (n = 210, 80.5%). IfR statements commonly addressed 'intervention' (n = 121, 57.6%), 'patient ' (n = 113, 53.8%), and 'study design' (n = 107, 51.0%). The most frequent PICO and study design combinations were 'patient and intervention ' (n = 71, 33.8%) and 'patient, intervention and study design ' (n = 34, 16.2%). Concepts underlying GRADE domains were rarely used for informing IfR recommendations: 'risk of bias ' (n = 2, 1.0%), and 'imprecision ' (n = 1, 0.5%), 'inconsistency ' (n = 1, 0.5%). Additional elements informing IfR were considerations on cost effectiveness (n = 9, 4.3%), reporting standards (n = 4, 1.9%), and individual patient data meta-analysis (n = 4, 1.9%).

CONCLUSION

Although about 80% of SRs included an IfR statement, the reporting of PICO elements varied across SRs. Concepts underlying GRADE domains were rarely used to derive IfR. Further work needs to assess the generalizability beyond SRs in advanced cancer patients. We suggest that more specific guidance on which and how IfR elements to report in SRs of interventions needs to be developed. Utilizing PICO elements and concepts underlying GRADE according to the Cochrane Handbook to state IfR seems to be a reasonable approach in the interim.

REGISTRATION

CRD42019134904.

First-pass effect in posterior acute ischemic stroke undergoing endovascular thrombectomy: A systematic review and meta-analysis

OBJECTIVES

First-pass effect (FPE) has been shown to be a predictor of favorable clinical outcomes following endovascular thrombectomy (EVT) for acute ischemic stroke (AIS) in the anterior circulation. Literature regarding FPE for posterior circulation AIS is sparse; we conducted a systematic review and meta-analysis to explore FPE in posterior circulation stroke undergoing EVT.

MATERIALS AND METHODS

We conducted a systematic review of the English literature in PubMed, Embase, Scopus, and Web of Science. FPE was defined as thrombolysis in cerebral infarction (TICI) 2c-3 and modified FPE (mFPE) was defined as TICI 2b-3 in one pass. Definitions of non-FPE and non-mFPE varied among studies. The primary outcome of interest was modified Rankin Scale (mRS) 0-2. Secondary outcomes of interest were mRS 0-3, symptomatic intracranial hemorrhage (sICH), and mortality. We calculated odds ratios (OR) and corresponding 95% confidence intervals (CI). Heterogeneity was assessed with Q statistic and I2 test.

RESULTS

Seven studies with 417 patients in the mFPE group, 942 in the non-mFPE group, 545 in the FPE group, and 1023 in the non-FPE group were included. Overall, FPE was associated with greater rates of 90-day mRS 0-2 (OR= 2.78, 95% CI= 2.11-3.65; P-value< 0.001) and mRS 0-3 (OR= 2.67, 95% CI= 1.98-3.60; P-value< 0.001); however, there was significant heterogeneity among studies for both mRS 0-2 (I2= 69%; P-value< 0.001) and mRS 0-3 (I2= 69%; P-value< 0.001). FPE and non-FPE were associated with similar rates of sICH (OR= 0.65, 95% CI= 0.40-1.07; P-value= 0.09), and no heterogeneity was observed (I2= 0%; P-value= 0.95). FPE was associated with lower rates of mortality (OR= 0.44, 95% CI= 0.33-0.58; P-value< 0.001), although heterogeneity was observed (I2= 58%; P-value= 0.01).

CONCLUSIONS

FPE is associated with favorable clinical outcomes in patients undergoing EVT for posterior circulation AIS. Future studies should work to further quantify the impact of FPE on outcomes in the posterior circulation.

Endovascular thrombectomy for ischemic stroke with large core volume: An updated, post-TESLA systematic review and meta-analysis of the randomized trials

BACKGROUND

Limited randomized controlled trials (RCTs) have been performed comparing endovascular thrombectomy (EVT) to medical therapy (MEDT) for acute ischemic stroke with extensive baseline ischemic injury (AIS-EBI). We conducted a systematic review and meta-analysis of RCTs reporting EVT for AIS-EBI.

METHODS

Using the Nested Knowledge AutoLit software, we conducted a systematic literature review from inception to 12 February 2023 within Web of Science, Embase, Scopus, and PubMed databases. Results of the TESLA trial were included on 10 June 2023. We included RCTs that compared EVT to MEDT for AIS with large ischemic core volume. The primary outcome of interest was a modified Rankin Score (mRS) 0-2. Secondary outcomes of interest included early neurological improvement (ENI), mRS 0-3, thrombolysis in cerebral infarction (TICI) 2b-3, symptomatic intracranial hemorrhage (sICH), and mortality. A random-effects model was used to calculate risk ratios (RRs) and their corresponding 95% confidence intervals (CIs).

RESULTS

We included four RCTs with 1310 patients, 661 of whom underwent EVT and 649 of whom were treated with MEDT. EVT was associated with greater rates of mRS 0-2 (RR = 2.33, 95% CI = 1.75-3.09; P-value < 0.001), mRS 0-3 (RR = 1.68, 95% CI = 1.33-2.12; P-value < 0.001), and ENI (RR = 2.24, 95% CI = 1.55-3.24; P-value < 0.001). Rates of sICH (RR = 1.99, 95% CI = 1.07-3.69; P-value = 0.03) were greater in the EVT group. Mortality (RR = 0.98, 95% CI = 0.83-1.15; P-value = 0.79) was comparable between the EVT and MEDT groups. The rate of successful reperfusion in the EVT group was 79.9% (95% CI = 75.6-83.6).

CONCLUSIONS

Although the rate of sICH was greater in the EVT group, EVT conferred a greater clinical benefit to MEDT for AIS-EBI based on available RCTs.

CT Perfusion vs Noncontrast CT for Late Window Stroke Thrombectomy: A Systematic Review and Meta-analysis

BACKGROUND AND OBJECTIVES

Patients with acute ischemic stroke (AIS) treated with endovascular thrombectomy (EVT) in the late window (6-24 hours) can be evaluated with CT perfusion (CTP) or with noncontrast CT (NCCT) only. Whether outcomes differ depending on the type of imaging selection is unknown. We conducted a systematic review and meta-analysis comparing outcomes between CTP and NCCT for EVT selection in the late therapeutic window.

METHODS

This study is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses 2020 guidelines. A systematic literature review of the English language literature was conducted using Web of Science, Embase, Scopus, and PubMed databases. Studies focusing on late-window AIS undergoing EVT imaged through CTP and NCCT were included. Data were pooled using a random-effects model. The primary outcome of interest was rate of functional independence, defined as modified Rankin scale 0-2. The secondary outcomes of interest included rates of successful reperfusion, defined as thrombolysis in cerebral infarction 2b-3, mortality, and symptomatic intracranial hemorrhage (sICH).

RESULTS

Five studies with 3,384 patients were included in our analysis. There were comparable rates of functional independence (odds ratio [OR] 1.03, 95% CI 0.87-1.22; p = 0.71) and sICH (OR 1.09, 95% CI 0.58-2.04; p = 0.80) between the 2 groups. Patients imaged with CTP had higher rates of successful reperfusion (OR 1.31, 95% CI 1.05-1.64; p = 0.015) and lower rates of mortality (OR 0.79, 95% CI 0.65-0.96; p = 0.017).

DISCUSSION

Although recovery of functional independence after late-window EVT was not more common in patients selected by CTP when compared with patients selected by NCCT only, patients selected by CTP had lower mortality.