Meta-epidemiological review identified variable reporting and handling of time-to-event analyses in publications of trials included in meta-analyses of systematic reviews

OBJECTIVES

Previous findings indicate limited reporting of systematic reviews with meta-analyses of time-to-event (TTE) outcomes. We assessed corresponding available information in trial publications included in such meta-analyses.

STUDY DESIGN AND SETTING

We extracted data from all randomized trials in pairwise, hazard ratio (HR)-based meta-analyses of primary outcomes and overall survival of 50 systematic reviews systematically identified from the Cochrane Database and Core Clinical Journals. Data on methods and characteristics relevant for TTE analysis of reviews, trials, and outcomes were extracted.

RESULTS

Meta-analyses included 235 trials with 315 trial analyses. Most prominently assessed was overall survival (91%). Definitions (61%), censoring reasons (41%), and follow-up specifications (56%) for trial outcomes were often missing. Available TTE data per trial were most frequently survival curves (83%), log-rank P values (76%), and HRs (72%). When trial TTE data recalculation was reported, reviews mostly specified HRs or P values (each 5%). Reviews primarily included intention-to-treat analyses (64%) and analyses not adjusted for covariates (25%). Except for missing outcome data, TTE-relevant trial characteristics, for example, informative censoring, treatment switching, and proportional hazards, were sporadically addressed in trial publications. Reporting limitations in trial publications translate to the review level.

CONCLUSION

TTE (meta)-analyses, in trial and review publications, need clear reporting standards.

GRADE concept 4: rating the certainty of evidence when study interventions or comparators differ from PICO targets

OBJECTIVES

This Grading of Recommendations Assessment, Development and Evaluation (GRADE) concept article offers systematic reviewers, guideline authors, and other users of evidence assistance in addressing randomized trial situations in which interventions or comparators differ from those in the target people, interventions, comparators, and outcomes. To clarify what GRADE considers under indirectness of interventions and comparators, we focus on a particular example: when comparator arm participants receive some or all aspects of the intervention management strategy (treatment switching).

STUDY DESIGN AND SETTING

An interdisciplinary panel of the GRADE working group members developed this concept article through an iterative review of examples in multiple teleconferences, small group sessions, and e-mail correspondence. After presentation at a GRADE working group meeting in November 2022, attendees approved the final concept paper, which we support with examples from systematic reviews and individual trials.

RESULTS

In the presence of safeguards against risk of bias, trials provide unbiased estimates of the effect of an intervention on the people as enrolled, the interventions as implemented, the comparators as implemented, and the outcomes as measured. Within the GRADE framework, differences in the people, interventions, comparators, and outcomes elements between the review or guideline recommendation targets and the trials as implemented constitute issues of indirectness. The intervention or comparator group management strategy as implemented, when it differs from the target comparator, constitutes one potential source of indirectness: Indirectness of interventions and comparators-comparator group receipt of the intervention constitutes a specific subcategory of said indirectness. The proportion of comparator arm participants that received the intervention and the apparent magnitude of effect bear on whether one should rate down, and if one does, to what extent.

CONCLUSION

Treatment switching and other differences between review or guideline recommendation target interventions and comparators vs. interventions and comparators as implemented in otherwise relevant trials are best considered issues of indirectness.

First-line therapy for adults with advanced renal cell carcinoma: a systematic review and network meta-analysis

BACKGROUND

Since the approval of tyrosine kinase inhibitors, angiogenesis inhibitors and immune checkpoint inhibitors, the treatment landscape for advanced renal cell carcinoma (RCC) has changed fundamentally. Today, combined therapies from different drug categories have a firm place in a complex first-line therapy. Due to the large number of drugs available, it is necessary to identify the most effective therapies, whilst considering their side effects and impact on quality of life (QoL).

OBJECTIVES

To evaluate and compare the benefits and harms of first-line therapies for adults with advanced RCC, and to produce a clinically relevant ranking of therapies. Secondary objectives were to maintain the currency of the evidence by conducting continuous update searches, using a living systematic review approach, and to incorporate data from clinical study reports (CSRs).

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, conference proceedings and relevant trial registries up until 9 February 2022. We searched several data platforms to identify CSRs.

SELECTION CRITERIA

We included randomised controlled trials (RCTs) evaluating at least one targeted therapy or immunotherapy for first-line treatment of adults with advanced RCC. We excluded trials evaluating only interleukin-2 versus interferon-alpha as well as trials with an adjuvant treatment setting. We also excluded trials with adults who received prior systemic anticancer therapy if more than 10% of participants were previously treated, or if data for untreated participants were not separately extractable.

DATA COLLECTION AND ANALYSIS

All necessary review steps (i.e. screening and study selection, data extraction, risk of bias and certainty assessments) were conducted independently by at least two review authors. Our outcomes were overall survival (OS), QoL, serious adverse events (SAEs), progression-free survival (PFS), adverse events (AEs), the number of participants who discontinued study treatment due to an AE, and the time to initiation of first subsequent therapy. Where possible, analyses were conducted for the different risk groups (favourable, intermediate, poor) according to the International Metastatic Renal-Cell Carcinoma Database Consortium Score (IMDC) or the Memorial Sloan Kettering Cancer Center (MSKCC) criteria. Our main comparator was sunitinib (SUN). A hazard ratio (HR) or risk ratio (RR) lower than 1.0 is in favour of the experimental arm.

MAIN RESULTS

We included 36 RCTs and 15,177 participants (11,061 males and 4116 females). Risk of bias was predominantly judged as being 'high' or 'some concerns' across most trials and outcomes. This was mainly due to a lack of information about the randomisation process, the blinding of outcome assessors, and methods for outcome measurements and analyses. Additionally, study protocols and statistical analysis plans were rarely available. Here we present the results for our primary outcomes OS, QoL, and SAEs, and for all risk groups combined for contemporary treatments: pembrolizumab + axitinib (PEM+AXI), avelumab + axitinib (AVE+AXI), nivolumab + cabozantinib (NIV+CAB), lenvatinib + pembrolizumab (LEN+PEM), nivolumab + ipilimumab (NIV+IPI), CAB, and pazopanib (PAZ). Results per risk group and results for our secondary outcomes are reported in the summary of findings tables and in the full text of this review. The evidence on other treatments and comparisons can also be found in the full text. Overall survival (OS) Across risk groups, PEM+AXI (HR 0.73, 95% confidence interval (CI) 0.50 to 1.07, moderate certainty) and NIV+IPI (HR 0.69, 95% CI 0.69 to 1.00, moderate certainty) probably improve OS, compared to SUN, respectively. LEN+PEM may improve OS (HR 0.66, 95% CI 0.42 to 1.03, low certainty), compared to SUN. There is probably little or no difference in OS between PAZ and SUN (HR 0.91, 95% CI 0.64 to 1.32, moderate certainty), and we are uncertain whether CAB improves OS when compared to SUN (HR 0.84, 95% CI 0.43 to 1.64, very low certainty). The median survival is 28 months when treated with SUN. Survival may improve to 43 months with LEN+PEM, and probably improves to: 41 months with NIV+IPI, 39 months with PEM+AXI, and 31 months with PAZ. We are uncertain whether survival improves to 34 months with CAB. Comparison data were not available for AVE+AXI and NIV+CAB. Quality of life (QoL) One RCT measured QoL using FACIT-F (score range 0 to 52; higher scores mean better QoL) and reported that the mean post-score was 9.00 points higher (9.86 lower to 27.86 higher, very low certainty) with PAZ than with SUN. Comparison data were not available for PEM+AXI, AVE+AXI, NIV+CAB, LEN+PEM, NIV+IPI, and CAB. Serious adverse events (SAEs) Across risk groups, PEM+AXI probably increases slightly the risk for SAEs (RR 1.29, 95% CI 0.90 to 1.85, moderate certainty) compared to SUN. LEN+PEM (RR 1.52, 95% CI 1.06 to 2.19, moderate certainty) and NIV+IPI (RR 1.40, 95% CI 1.00 to 1.97, moderate certainty) probably increase the risk for SAEs, compared to SUN, respectively. There is probably little or no difference in the risk for SAEs between PAZ and SUN (RR 0.99, 95% CI 0.75 to 1.31, moderate certainty). We are uncertain whether CAB reduces or increases the risk for SAEs (RR 0.92, 95% CI 0.60 to 1.43, very low certainty) when compared to SUN. People have a mean risk of 40% for experiencing SAEs when treated with SUN. The risk increases probably to: 61% with LEN+PEM, 57% with NIV+IPI, and 52% with PEM+AXI. It probably remains at 40% with PAZ. We are uncertain whether the risk reduces to 37% with CAB. Comparison data were not available for AVE+AXI and NIV+CAB.

AUTHORS' CONCLUSIONS

Findings concerning the main treatments of interest comes from direct evidence of one trial only, thus results should be interpreted with caution. More trials are needed where these interventions and combinations are compared head-to-head, rather than just to SUN. Moreover, assessing the effect of immunotherapies and targeted therapies on different subgroups is essential and studies should focus on assessing and reporting relevant subgroup data. The evidence in this review mostly applies to advanced clear cell RCC.

Chimeric antigen receptor (CAR) T-cell therapy for people with relapsed or refractory diffuse large B-cell lymphoma

BACKGROUND

Diffuse large B-cell lymphoma (DLBCL) is an aggressive cancer of the lymphatic system. About 30% to 40% of people with DLBCL experience relapse and 10% are refractory to first-line treatment usually consisting of R-CHOP chemotherapy. Of those eligible for second-line treatment, commonly consisting of salvage chemotherapy followed by autologous stem-cell transplantation (ASCT), around 50% experience relapse. With a median overall survival of less than six to 12 months, the prognosis of individuals who relapse or are refractory (r/r) to advanced lines of treatment or of those who are ineligible for ASCT, is very poor. With the introduction of chimeric antigen receptor (CAR) T-cell therapy, a novel treatment option for these people is available.

OBJECTIVES

To assess the benefits and harms of chimeric antigen receptor (CAR) T-cell therapy for people with relapsed or refractory (r/r) DLBCL.

SEARCH METHODS

An experienced information specialist performed a systematic database search for relevant articles on CENTRAL, MEDLINE and Embase until September 11th, 2020. We also searched trial registries and reference lists of identified studies up to this date. All search results were screened by two authors independently and a third author was involved in case of discrepancies.

SELECTION CRITERIA

We included prospectively planned trials evaluating CAR T-cell therapy for people with r/r DLBCL. We had planned to include randomised controlled trials (RCTs) and we flexibly adapted eligibility criteria to the most reliable study designs available. We excluded studies involving fewer than 10 participants with r/r DLBCL and studies with a proportion of participants with r/r DLBCL below 70%, unless data were reported separately for this subgroup.

DATA COLLECTION AND ANALYSIS

Two review authors extracted data and performed risk of bias ratings independently. A third author was involved in case of disagreements. As our search did not yield any completed RCTs, prospective controlled non-randomised studies of interventions (NRSIs) or prospective observational studies with a control group, we did not meta-analyse data and reported all results narratively. We adopted the GRADE approach to assess the certainty of the evidence for prioritised outcomes.

MAIN RESULTS

We identified 13 eligible uncontrolled studies evaluating a single or multiple arms of CAR T-cell therapies. We also identified 38 ongoing studies, including three RCTs. Ten studies are awaiting classification due to completion with no retrievable results data or insufficient data to justify inclusion. The mean number of participants enrolled, treated with CAR T-cell therapy and evaluated in the included studies were 79 (range 12 to 344; data unavailable for two studies), 61 (range 12 to 294; data unavailable for one study) and 52 (range 11 to 256), respectively. Most studies included people with r/r DLBCL among people with other haematological B-cell malignancies. Participants had received at least a median of three prior treatment lines (data unavailable for four studies), 5% to 50% had undergone ASCT (data unavailable for five studies) and, except for two studies, 3% to 18% had undergone allogenic stem-cell transplantation (data unavailable for eight studies). The overall risk of bias was high for all studies, in particular, due to incomplete follow-up and the absence of blinding. None of the included studies had a control group so that no adequate comparative effect measures could be calculated. The duration of follow-up varied substantially between studies, in particular, for harms. Our certainty in the evidence is very low for all outcomes. Overall survival was reported by eight studies (567 participants). Four studies reported survival rates at 12 months which ranged between 48% and 59%, and one study reported an overall survival rate of 50.5% at 24 months. The evidence is very uncertain about the effect of CAR T-cell therapy on overall survival. Two studies including 294 participants at baseline and 59 participants at the longest follow-up (12 months or 18 months) described improvements of quality of life measured with the EuroQol 5-Dimension 5-Level visual analogue scale (EQ-5D-5L VAS) or Function Assessment of Cancer Therapy-Lymphoma (FACT-Lym). The evidence is very uncertain about the effect of CAR T-cell therapy on quality of life. None of the studies reported treatment-related mortality. Five studies (550 participants) reported the occurrence of adverse events among participants, ranging between 99% and 100% for any grade adverse events and 68% to 98% for adverse events grade ≥ 3. In three studies (253 participants), 56% to 68% of participants experienced serious adverse events, while in one study (28 participants), no serious adverse events occurred. CAR T-cell therapy may increase the risk of adverse events and serious adverse events but the evidence is very uncertain about the exact risk. The occurrence of cytokine release syndrome (CRS) was reported in 11 studies (675 participants) under use of various grading criteria. Five studies reported between 42% and 100% of participants experiencing CRS according to criteria described in Lee 2014. CAR T-cell therapy may increase the risk of CRS but the evidence is very uncertain about the exact risk. Nine studies (575 participants) reported results on progression-free survival, disease-free survival or relapse-free survival. Twelve-month progression-free survival rates were reported by four studies and ranged between 44% and 75%. In one study, relapse-free survival remained at a rate of 64% at both 12 and 18 months. The evidence is very uncertain about the effect of CAR T-cell therapy on progression-free survival. Thirteen studies (620 participants) provided data on complete response rates. At six months, three studies reported complete response rates between 40% and 45%. The evidence is very uncertain about the effect of CAR T-cell therapy on complete response rates.

AUTHORS' CONCLUSIONS

The available evidence on the benefits and harms of CAR T-cell therapy for people with r/r DLBCL is limited, mainly because of the absence of comparative clinical trials. The results we present should be regarded in light of this limitation and conclusions should be drawn very carefully. Due to the uncertainty in the current evidence, a large number of ongoing investigations and a risk of substantial and potentially life-threatening complications requiring supplementary treatment, it is critical to continue evaluating the evidence on this new therapy.

Association of Supporting Trial Evidence and Reimbursement for Off-Label Use of Cancer Drugs

IMPORTANCE

In many health systems, access to off-label drug use is controlled through reimbursement restrictions by health insurers, especially for expensive cancer drugs.

OBJECTIVE

To determine whether evidence from randomized clinical trials is associated with reimbursement decisions for requested off-label use of anticancer drugs in the Swiss health system.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study used reimbursement requests from routinely collected health records of 5809 patients with drug treatment for cancer between January 2015 and July 2018 in 3 major cancer centers, covering cancer care of approximately 5% of the Swiss population, to identify off-label drug use. For each off-label use indication with 3 or more requests, randomized clinical trial evidence on treatment benefits was systematically identified for overall survival (OS) or progression-free survival (PFS). Data were analyzed from August 2018 to December 2020.

EXPOSURES

Available randomized clinical trial evidence on benefits for OS or PFS for requested off-label use indications.

MAIN OUTCOMES AND MEASURES

The main outcome was the association between evidence for treatment benefit (expressed as improved OS or PFS) and reimbursement in multivariable regression models.

RESULTS

Among 3046 patients with cancer, 695 off-label use reimbursement requests in 303 different indications were made for 598 patients (median [interquartile range] age, 64 [53-73] years; 420 [60%] men). Off-label use was intended as first-line treatment in 311 requests (45%). Reimbursement was accepted in 446 requests (64%). For 71 indications, including 431 requests for 376 patients, there were 3 or more requests. Of these, 246 requests (57%) had no supporting evidence for OS or PFS benefit. Reimbursement was granted in 162 of 246 requests without supporting evidence (66%). Of 117 requests supported by OS benefit, 79 (67%) were reimbursed, and of 68 requests supported by PFS benefit alone, 54 (79%) were reimbursed. Evidence of OS benefit from randomized clinical trials was not associated with a higher chance of reimbursement (odds ratio, 0.76, 95% CI, 0.45-1.27).

CONCLUSIONS AND RELEVANCE

These findings suggest that in a health care system enabling access to off-label use, it was frequently intended as a first-line treatment in cancer care. Availability of randomized clinical trial evidence showing survival benefit was not associated with reimbursement decisions for off-label anticancer drug treatment in Switzerland. A transparent process with criteria considering clinical evidence is needed for evidence-based reimbursement decisions to ensure fair access to cancer treatments.

["Transferability": Is GRADE the solution?]

Biological and social factors, such as age, comorbidities and the care system, may - as well-established effect-modifiers - limit the transferability of study results to populations with dissimilar characteristics. In order to enable transparent and evidence-based decisions in systematic reviews and guidelines targeting subpopulations that are little or not represented in the study landscape, the GRADE approach is a valid tool to assess the certainty of the evidence. GRADE provides a structured methodology that covers all steps, from developing a precise question, prioritizing patient-relevant outcomes and assessing the available evidence to derive recommendations for practice, among other things. Evaluating confidence in a body of evidence comprises judgments on risk of bias, study heterogeneity, directness, including comparability between study population and target population, precision of effect estimates and publication bias. Because GRADE demands transparent decisions about the applicability of study results from the study population to the target population, gaps in the evidence landscape can be uncovered. Overall, the approach cannot solve the problem of the transferability of study results. It does, however, support the explicit handling of applicability and can give impetus to targeted research gaps.

Nivolumab for adults with Hodgkin's lymphoma (a rapid review using the software RobotReviewer)

BACKGROUND

Hodgkin's lymphoma (HL) is a cancer of the lymphatic system, and involves the lymph nodes, spleen and other organs such as the liver, lung, bone or bone marrow, depending on the tumour stage. With cure rates of up to 90%, HL is one of the most curable cancers worldwide. Approximately 10% of people with HL will be refractory to initial treatment or will relapse; this is more common in people with advanced stage or bulky disease. Standard of care for these people is high-dose chemotherapy and autologous stem cell transplantation (ASCT), but only 55% of participants treated with high-dose chemotherapy and ASCT are free from treatment failure at three years, with an overall survival (OS) of about 80% at three years.Checkpoint inhibitors that target the interaction of the programmed death (PD)-1 immune checkpoint receptor, and its ligands PD-L1 and PD-L2, have shown remarkable activity in a wide range of malignancies. Nivolumab is an anti-(PD)-1 monoclonal antibody and currently approved by the US Food and Drug Administration (FDA) for the treatment of melanoma, non-small cell lung cancer, renal cell carcinoma and, since 2016, for classical Hodgkin's lymphoma (cHL) after treatment with ASCT and brentuximab vedotin.

OBJECTIVES

To assess the benefits and harms of nivolumab in adults with HL (irrespective of stage of disease).

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, International Pharmaceutical Abstracts, conference proceedings and six study registries from January 2000 to May 2018 for prospectively planned trials evaluating nivolumab.

SELECTION CRITERIA

We included prospectively planned trials evaluating nivolumab in adults with HL. We excluded trials in which less than 80% of participants had HL, unless the trial authors provided the subgroup data for these participants in the publication or after we contacted the trial authors.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data and assessed potential risk of bias. We used the software RobotReviewer to extract data and compared results with our findings. As we did not identify any randomised controlled trials (RCTs) or non-RCTs, we did not meta-analyse data.

MAIN RESULTS

Our search found 782 potentially relevant references. From these, we included three trials without a control group, with 283 participants. In addition, we identified 14 ongoing trials evaluating nivolumab, of which two are randomised. Risk of bias of the three included studies was moderate to high. All of the participants were in relapsed stage, most of them were heavily pretreated and had received at least two previous treatments, most of them had also undergone ASCT. As we did not identify any RCTs, we could not use the software RobotReviewer to assess risk of bias. The software identified correctly that one study was not an RCT and did not extract any trial data, but extracted characteristics of the other two studies (although also not RCTs) in a sufficient way.Two studies with 260 participants evaluated OS. After six months, OS was 100% in one study and median OS (the timepoint when only 50% of participants were alive) was not reached in the other trial after a median follow-up of 18 months (interquartile range (IQR) 15 to 22 months) (very low certainty evidence, due to observational trial design, heterogenous patient population in terms of pretreatments and various follow-up times (downgrading by 1 point)). In one study, one out of three cohorts reported quality of life. It was unclear whether there was an effect on quality of life as only a subset of participants filled out the follow-up questionnaire (very low certainty evidence). Three trials (283 participants) evaluated progression-free survival (PFS) (very low certainty evidence). Six-month PFS ranged between 60% and 86%, and median PFS ranged between 12 and 18 months. All three trials (283 participants) reported complete response rates, ranging from 12% to 29%, depending on inclusion criteria and participants' previous treatments (very low certainty evidence).One trial (243 participants) reported drug-related grade 3 or 4 adverse events (AEs) only after a median follow-up of 18 months (IQR 15 to 22 months); these were fatigue (23%), diarrhoea (15%), infusion reactions (14%) and rash (12%). The other two trials (40 participants) reported 23% to 52% grade 3 or 4 AEs after six months' follow-up (very low certainty evidence). Only one trial (243 participants) reported drug-related serious AEs; 2% of participants developed infusion reactions and 1% pneumonitis (very low certainty evidence).None of the studies reported treatment-related mortality.

AUTHORS' CONCLUSIONS

To date, data on OS, quality of life, PFS, response rate, or short- and long-term AEs are available from small uncontrolled trials only. The three trials included heavily pretreated participants, which had previously undergone regimens of BV or ASCT. For these participants, median OS was not reached after follow-up times of at least 16 months (more than 50% of participants with a limited life expectancy were alive at this timepoint). Only one cohort out of three only reported quality of life, with limited follow-up data so that meaningful conclusions were not possible. Serious adverse events occurred rarely. Currently, data are too sparse to make a clear statement on nivolumab for people with relapsed or refractory HL except for heavily pretreated people, which had previously undergone regimens of BV or ASCT. When interpreting these results, it is important to consider that proper RCTs should confirm these findings.As there are 14 ongoing trials evaluating nivolumab, of which two are RCTs, it is possible that an update of this review will be published in the near future and that this update will show different results to those reported here.

A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer

OBJECTIVE

To compare cancer-related systematic reviews (SRs) published in the Cochrane Database of SRs (CDSR) and high-impact journals, with respect to type, content, quality and citation rates.

DESIGN

Methodological SR with assessment and comparison of SRs and meta-analyses. Two authors independently assessed methodological quality using an Assessment of Multiple Systematic Reviews (AMSTAR)-based extraction form. Both authors independently screened search results, extracted content-relevant characteristics and retrieved citation numbers of the included reviews using the Clarivate Analytics Web of Science database.

DATA SOURCES

Cancer-related SRs were retrieved from the CDSR, as well as from the 10 journals which publish oncological SRs and had the highest impact factors, using a comprehensive search in both the CDSR and MEDLINE.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included all cancer-related SRs and meta-analyses published from January 2011 to May 2016. Methodological SRs were excluded.

RESULTS

We included 346 applicable Cochrane reviews and 215 SRs from high-impact journals. Cochrane reviews consistently met more individual AMSTAR criteria, notably with regard to an a priori design (risk ratio (RR) 3.89; 95% CI 3.10 to 4.88), inclusion of the grey literature and trial registries (RR 3.52; 95% CI 2.84 to 4.37) in their searches, and the reporting of excluded studies (RR 8.80; 95% CI 6.06 to 12.78). Cochrane reviews were less likely to address questions of prognosis (RR 0.04; 95% CI 0.02 to 0.09), use individual patient data (RR 0.03; 95% CI 0.01 to 0.09) or be based on non-randomised controlled trials (RR 0.04; 95% CI 0.02 to 0.09). Citation rates of Cochrane reviews were notably lower than those for high-impact journals (Cochrane reviews: mean number of citations 6.52 (range 0-143); high-impact journal SRs: 74.45 (0-652)).

CONCLUSIONS

When comparing cancer-related SRs published in the CDSR versus those published in high-impact medical journals, Cochrane reviews were consistently of higher methodological quality, but cited less frequently.