A comparison between the assessments of progression-free survival by local investigators versus blinded independent central reviews in phase III oncology trials

Empirical evidence of observer bias in randomized clinical trials: updated and expanded analysis of trials with both blinded and non-blinded outcome assessors

OBJECTIVES

To study the impact of lack of blinding of outcome assessors on estimated treatment effects of randomized clinical trials.

STUDY DESIGN AND SETTING

Meta-epidemiological study. We included randomized trials with binary or measurement scale outcomes that (1) allocated patients to subtrials with and without blinded outcome assessment, or (2) had both blinded and non-blinded assessments of the same outcome. We identified trials from previous meta-epidemiological studies and searched six databases from 2013 to 2024. We calculated a ratio of odds ratios (ROR) for each trial. A ROR < 1 indicated a more favorable effect estimate by the non-blinded assessor. We pooled RORs using random effects meta-analysis and conducted meta-regression, subgroup, and sensitivity analyses.

RESULTS

We included 66 trials (9451 patients) across 18 clinical specialties. The pooled RORs in 43 trials (7055 patients) was 0.71 (0.55-0.92). Thirty of the 43 trials assessed highly subjective outcomes. Meta-regression showed no statistically significant association between ROR and scores for outcome subjectivity (P = .53), vulnerability (P = .91), and involvement (P = .99). Heterogeneity was partly explained by a larger impact in non-drug trials, ROR 0.62 (0.46-0.84), and industry-funded trials, ROR 0.57 (0.37-0.88). Sensitivity analyses, including imputed data for 23 trials (2396 patients randomized), did not modify the observed impact importantly.

CONCLUSION

We provide empirical evidence of considerable bias in effect estimates of randomized trials with non-blinded assessors of subjective binary and measurement scale outcomes. Non-blinded assessors exaggerated effect estimates, expressed as odds ratios, by 29% (8%-45%) on average. This strongly supports blinding outcome assessors of subjective outcomes.

PLAIN LANGUAGE SUMMARY

In a randomized clinical trial, the person evaluating the results (assessor) may be either unaware of the intervention received by participants (blinded assessor) or aware of it (non-blinded assessor). Knowing which treatment a patient received can influence the assessor's evaluation of the effect; for example, if an assessor has high expectations for a new experimental intervention, they may rate a patient's improvement more favorably in the group that received the intervention compared to the group that did not. We call this observer bias. In this study, we compared the results obtained from blinded assessors to those from non-blinded assessors within the same trials to estimate the impact of observer bias in randomized trials. We found that non-blinded assessors exaggerated the experimental intervention effect by approximately 29%, on average, compared to blinded assessors. Our results indicate that when an evaluation of a patient in a randomized trial requires judgment, there is potential for substantial bias if assessors are not blinded. To ensure more reliable results, randomized clinical trials should blind assessors whenever possible.

Blinded independent central review versus local investigator assessment of PFS in RCTs of immunotherapy in advanced cancers: A systematic review and meta-analysis

BACKGROUND

Assessment of Progression-free survival (PFS) events by investigators might be inaccurate in randomized controlled trials (RCTs) with open-label design. We explored differences in PFS evaluated by blinded independent central review (BICR) or local investigator assessment (IA) in trials testing immunotherapy (IO) in advanced cancers.

METHODS

We systematically reviewed articles of RCTs investigating IO in advanced tumors, published in PubMed-indexed journals up to December 2023. For each RCT, we collected PFS results by BICR and by local IA. We calculated the discrepancy index (DI) as the ratio of BICR and IA Hazard Ratios (HRBICR/HRIA) for PFS. An overall DI and relative confidence interval (CI) were calculated using a fixed-effect model weighted for the inverse of variance.

FINDINGS

Only 32/140 (22.9 %) RCTs reported both BICR and local IA PFS data, including 17,054 patients. PFS was the sole primary endpoint in 19/32 (59.4 %) and a co-primary endpoint 9/32 (28.2 %) trials. The study design was open label or double-blind in 17/32 (53.1 %) and 15/32 (46.9 %) RCTs, respectively. The overall DI was 1.07 (95 % CI 1.01-1.13; I2 =0, p = 0.02), revealing a more optimistic analysis of results in favor of local IA. In the 17 open-label trials, the overall DI was 1.09 (95 % CI 1.02-1.17, I2 =0, p = 0.02), revealing a more favorable interpretation of PFS results by local investigators.

INTERPRETATION

We found a statistically significant difference between BICR and local IA of PFS in trials of IO in cancer. These results suggest that the double assessment is recommended in RCTs testing IO, especially in open-label trials.

FUNDING

This work was supported by the MFAG27826-2022 grant (Dr. Alberto Servetto).

Pembrolizumab plus chemotherapy for first-line treatment of advanced triple-negative breast cancer

Aim: A systematic review and network meta-analysis (NMA) was performed to evaluate the efficacy of first-line treatments for locally recurrent unresectable or metastatic triple-negative breast cancer (TNBC) patients.Materials & methods: Databases were searched for randomized controlled trials evaluating first-line treatments for locally recurrent unresectable or metastatic TNBC patients. NMA was performed to estimate relative treatment effects on overall and progression-free survival between pembrolizumab + chemotherapy and other interventions.Results: NMA including eight trials showed that the relative efficacy of pembrolizumab + chemotherapy was statistically superior to that of other immunotherapy- or chemotherapy-based treatment regimens.Conclusion: Pembrolizumab + chemotherapy confers benefits in survival outcomes versus alternative interventions for the first-line treatment of locally recurrent unresectable or metastatic TNBC patients.

Compelling evidence from meta-epidemiological studies demonstrates overestimation of effects in randomized trials that fail to optimize randomization and blind patients and outcome assessors

OBJECTIVES

To investigate the impact of potential risk of bias elements on effect estimates in randomized trials.

STUDY DESIGN AND SETTING

We conducted a systematic survey of meta-epidemiological studies examining the influence of potential risk of bias elements on effect estimates in randomized trials. We included only meta-epidemiological studies that either preserved the clustering of trials within meta-analyses (compared effect estimates between trials with and without the potential risk of bias element within each meta-analysis, then combined across meta-analyses; between-trial comparisons), or preserved the clustering of substudies within trials (compared effect estimates between substudies with and without the element, then combined across trials; within-trial comparisons). Separately for studies based on between- and within-trial comparisons, we extracted ratios of odds ratios (RORs) from each study and combined them using a random-effects model. We made overall inferences and assessed certainty of evidence based on Grading of Recommendations, Assessment, development, and Evaluation and Instrument to assess the Credibility of Effect Modification Analyses.

RESULTS

Forty-one meta-epidemiological studies (34 of between-, 7 of within-trial comparisons) proved eligible. Inadequate random sequence generation (ROR 0.94, 95% confidence interval [CI] 0.90-0.97) and allocation concealment (ROR 0.92, 95% CI 0.88-0.97) probably lead to effect overestimation (moderate certainty). Lack of patients blinding probably overestimates effects for patient-reported outcomes (ROR 0.36, 95% CI 0.28-0.48; moderate certainty). Lack of blinding of outcome assessors results in effect overestimation for subjective outcomes (ROR 0.69, 95% CI 0.51-0.93; high certainty). The impact of patients or outcome assessors blinding on other outcomes, and the impact of blinding of health-care providers, data collectors, or data analysts, remain uncertain. Trials stopped early for benefit probably overestimate effects (moderate certainty). Trials with imbalanced cointerventions may overestimate effects, while trials with missing outcome data may underestimate effects (low certainty). Influence of baseline imbalance, compliance, selective reporting, and intention-to-treat analysis remain uncertain.

CONCLUSION

Failure to ensure random sequence generation or adequate allocation concealment probably results in modest overestimates of effects. Lack of patients blinding probably leads to substantial overestimates of effects for patient-reported outcomes. Lack of blinding of outcome assessors results in substantial effect overestimation for subjective outcomes. For other elements, though evidence for consistent systematic overestimate of effect remains limited, failure to implement these safeguards may still introduce important bias.

Progression-free survival assessment by local investigators versus blinded independent central review in randomized clinical trials in metastatic breast cancer: A systematic review and meta-analysis

INTRODUCTION

In randomized clinical trials (RCTs), blinded independent central review (BICR) is used to minimize heterogeneity and bias associated with radiological response evaluation by local investigators. However, BICR adds costs and complexity to the trial management. We assessed the discrepancy index between progression-free survival (PFS) assessment by local investigators and by BICR in RCTs conducted in patients with metastatic breast cancer (MBC).

METHODS

A systematic search of PubMed, Embase, Cochrane databases and conference proceedings (ASCO, SABCS, ESMO) was performed up to January 4, 2023 (PROSPERO: CRD42021229865). All RCTs published from 2000 to 2022, including MBC patients treated in first- or second-line, and reporting PFS assessed by local investigators and BICR were included. A discrepancy index between BICR-assessed and investigator-assessed HR was calculated for each trial and an overall combined DI was obtained using a fixed-effects model. The agreement between hazard ratios (HR) of PFS assessed by local investigators and BICR was measured using intraclass correlation coefficient (ICC).

RESULTS

We analyzed 24 studies including 13,168 patients. Among them, 19 (79%) were in first-line, 18 (75%) were phase III trials and 23 (96%) had PFS as primary endpoint. The overall combined discrepancy index was 0.97 (95%CI 0.85-1.10; ICC 0.831, p < 0.001) suggesting no statistically significant difference in PFS assessment between local investigators and BICR. This result was consistent across all analyzed subgroups.

CONCLUSIONS

The good concordance between local investigator and BICR assessments supports the reliability of local investigator-assessed PFS as primary endpoint for RCTs in MBC and questions the practical utility of implementing BICR in all RCTs.

Detection bias in open-label trials of anticancer drugs: a meta-epidemiological study

OBJECTIVES

In anticancer clinical trials, particularly open-label trials, central reviewers are recommended to evaluate progression-free survival (PFS) and objective response rate (ORR) to avoid detection bias of local investigators. However, it is not clear whether the bias has been adequately identified, or to what extent it consistently distorts the results. Therefore, the objective of this study was to evaluate the detection bias in oncological open-label trials by confirming whether local investigators overestimate the PFS and ORR compared with the findings of central reviewers.

DESIGN

Meta-epidemiological study.

DATA SOURCES

MEDLINE via PubMed from 1 January 2010 to 30 June 2021.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Open-label, parallel-group superiority, randomised trials of anticancer drugs that adjudicated PFS or ORR by both central reviewers and local investigators.

REVIEW METHODS

We assessed the values for the same outcome (PFS and ORR) adjudicated by both central reviewers and local investigators. A random-effects model was used to estimate the ratio of HR (RHR) for PFS and the ratio of OR (ROR) for ORR between central reviewers and local investigators. An RHR lower than 1 and an ROR higher than 1 indicated an overestimation of the effect estimated by local investigators.

RESULTS

We retrieved 1197 records of oncological open-label trials after full-text screening. We identified 171 records (PFS: 149 records, ORR: 136 records) in which both central reviewers and local investigators were used, and included 114 records (PFS: 92 records, ORR: 74 records) for meta-analyses. While the RHR for PFS was 0.95 (95% CI 0.91 to 0.98), the ROR of ORR was 1.00 (95% CI 0.91 to 1.09). The results remained unchanged in the prespecified sensitivity analysis.

CONCLUSIONS

This meta-epidemiological study found that overestimation of local investigators has a small impact on evaluating PFS and ORR in oncological open-label trials. However, a limitation of this study is that it did not include data from all trials; hence, the results may not fully evaluate detection bias. The necessity of central reviewers in oncological open-label trials needs to be assessed by further studies that overcome this limitation.

TRIAL REGISTRATION NUMBER

CTR-UMIN000044623.

Real-World Evaluation of Quality of Life, Effectiveness, and Safety of Aflibercept Plus FOLFIRI in Patients with Metastatic Colorectal Cancer: The Prospective QoLiTrap Study

Aflibercept plus FOLFIRI prolongs overall survival (OS) in patients with metastatic colorectal cancer after the failure of oxaliplatin-containing therapy. QoLiTrap prospectively evaluated the quality of life (QoL) and effectiveness of this regimen in daily clinical practice, according to RAS status, sex, and prior targeted therapy, especially epidermal growth factor receptor inhibitors (EGFR-I). The primary endpoint was the percentage of patients whose EORTC QLQ-C30 global health status (GHS) improved or reduced by <5% from baseline during the first 12 weeks of therapy. Secondary endpoints included objective response rate (ORR), progression-free survival (PFS), overall survival (OS), and safety. One thousand two hundred and seventy-seven patients were treated with aflibercept plus FOLFIRI and 872 were evaluable for QoL. GHS improved or decreased by <5% in 40.3% of cases. The ORR was 20.8%, the median PFS was 7.8 months (95% confidence interval (CI), 7.3−8.3), and the median OS was 14.4 months (95% CI, 13.1−18.1). After prior EGFR-I, the ORR was 23.7%, median PFS was 9.4 months (95% CI, 6.5−12.9), and median OS was 17.4 months (95% CI, 10.5−33.7). The safety profile was consistent with previously reported data. Aflibercept plus FOLFIRI given in daily practice maintained QoL in mCRC patients, was associated with a high objective tumor response, and retained its activity regardless of sex, RAS status, and prior EGFR-I therapy.

Local Investigators Significantly Overestimate Overall Response Rates Compared to Blinded Independent Central Reviews in Uncontrolled Oncology Trials: A Comprehensive Review of the Literature

Several drugs gained market authorization based on the demonstration of improved progression-free survival (PFS), adopted as a primary endpoint in Phase 3 clinical trials. In addition, an increasing number of drugs have been granted accelerated approval, and sometimes regular approval, by the main regulatory agencies based on the evaluation of the overall response rate in Phase 1 and 2 clinical trials. However, while the overall survival is an unbiased measure of drug efficacy, these outcomes rely on the assessment of radiological images and patients’ categorization using standardized response criteria. The evaluation of these outcomes may be influenced by subjective factors, particularly when the analysis is performed locally. In fact, blinding of treatment is not always possible in modern oncology trials. Therefore, a blinded independent central review is often adopted to overcome the problem of expectation bias associated with local investigator assessments. In this regard, we have recently observed that local investigators tend to overestimate the overall response rate in comparison to central reviewers in Phase 2 clinical trials, whereas we did not find any significant evaluation bias between local investigators and central reviews when considering progression-free survival in both Phase 2 and 3 trials. In the present article, we have tried to understand the reasons behind this discrepancy by reviewing the available evidence in the literature. In addition, a further analysis of Phase 2 and 3 clinical trials that included the evaluation of both endpoints showed that local investigators significantly overestimate overall response rates compared to blinded independent central reviews in uncontrolled oncology trials.

Concordance of Clinician-Documented and Imaging Response in Patients With Stage IV Non-Small Cell Lung Cancer Treated With First-Line Therapy

IMPORTANCE

In observational oncology studies of solid tumors, response to treatment can be evaluated based on electronic health record (EHR) documentation (clinician-assessed response [CAR]), an approach different from standardized radiologist-measured response (Response Evaluation Criteria in Solid Tumours [RECIST] 1.1).

OBJECTIVE

To evaluate the feasibility of an imaging response based on RECIST (IRb-RECIST) and the concordance between CAR and imaging response based on RECIST assessments, and investigate discordance causes.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study used an EHR-derived, deidentified database that included patients with stage IV non-small cell lung cancer (NSCLC) diagnosed between January 1, 2011, to June 30, 2019, selected from 3 study sites. Data analysis was conducted in August, 2020.

EXPOSURES

Undergoing first-line therapy and imaging assessments of response to treatment.

MAIN OUTCOMES AND MEASURES

In this study, CAR assessments (referred to in prior publications as "real-world response" [rwR]) were defined as clinician-documented changes in disease burden at radiologic evaluation time points; they were abstracted manually and assigned to response categories. The RECIST-based assessments accommodated routine practice patterns by using a modified version of RECIST 1.1 (IRb-RECIST), with independent radiology reads. Concordance was calculated as the percent agreement across all response categories and across a dichotomous stratification (response [complete or partial] vs no response), unconfirmed or confirmed.

RESULTS

This study found that, in 100 patients evaluated for concordance, agreement between CAR and IRb-RECIST was 71% (95% CI, 61%-80%), and 74% (95% CI, 64%-82%) for confirmed and unconfirmed response, respectively. There were more responders using CAR than IRb-RECIST (40 vs 29 with confirmation; 64 vs 43 without confirmation). The main sources of discordance were the different use of thresholds for tumor size changes by RECIST vs routine care, and unavailable baseline or follow-up scans resulting in inconsistent anatomic coverage over time.

CONCLUSIONS AND RELEVANCE

In this cohort study of patients with stage IV NSCLC, we collected routine-care imaging, showing the feasibility of response evaluation using IRb-RECIST criteria with independent centralized review. Concordance between CAR and centralized IRb-RECIST was moderate. Future work is needed to evaluate the generalizability of these results to broader populations, and investigate concordance in other clinical settings.

Cemiplimab for Locally Advanced and Metastatic Cutaneous Squamous-Cell Carcinomas: Real-Life Experience from the French CAREPI Study Group

Simple Summary

Prognosis of advanced cutaneous squamous-cell carcinoma (CSCC) is poor. Recent clinical trials have shown that immunotherapy achieves significantly improved survival of patients with advanced CSCCs. However, few real-world data are available on treatment patterns and clinical outcomes of patients with advanced CSCCs receiving anti-programmed cell-death protein-1 (PD-1). To approach this issue, we conducted a retrospective study on 245 patients with advanced CSCCs from 58 centers who had been enrolled in an early-access program; 240 received cemiplimab. Our objectives were to evaluate, in the real-life setting, best overall response rate, progression-free survival, overall survival and safety. Results demonstrated cemiplimab efficacy in patients with advanced CSCCs, regardless of immune status. Patients with good Eastern Cooperative Oncology Group performance status benefited more from cemiplimab. The safety profile was acceptable.

Abstract

Although cemiplimab has been approved for locally advanced (la) and metastatic (m) cutaneous squamous-cell carcinomas (CSCCs), its real-life value has not yet been demonstrated. An early-access program enrolled patients with la/mCSCCs to receive cemiplimab. Endpoints were best overall response rate (BOR), progression-free survival (PFS), overall survival (OS), duration of response (DOR) and safety. The 245 patients (mean age 77 years, 73% male, 49% prior systemic treatment, 24% immunocompromised, 27% Eastern Cooperative Oncology Group performance status (PS) ≥ 2) had laCSCCs (35%) or mCSCCs (65%). For the 240 recipients of ≥1 infusion(s), the BOR was 50.4% (complete, 21%; partial, 29%). With median follow-up at 12.6 months, median PFS was 7.9 months, and median OS and DOR were not reached. One-year OS was 73% versus 36%, respectively, for patients with PS < 2 versus ≥ 2. Multivariate analysis retained PS ≥ 2 as being associated during the first 6 months with PFS and OS. Head-and-neck location was associated with longer PFS. Immune status had no impact. Severe treatment-related adverse events occurred in 9% of the patients, including one death from toxic epidermal necrolysis. Cemiplimab real-life safety and efficacy support its use for la/mCSCCs. Patients with PS ≥ 2 benefited less from cemiplimab, but it might represent an option for immunocompromised patients.

Progression-free survival by investigator versus blinded independent central review in newly diagnosed patients with high-grade serous ovarian cancer: Analysis of the VELIA/GOG-3005 trial

OBJECTIVE

In the phase 3 VELIA/GOG-3005 trial, veliparib added to carboplatin-paclitaxel and continued as maintenance improved progression-free survival (PFS) compared to carboplatin-paclitaxel alone in patients with newly diagnosed ovarian carcinoma. Primary analysis of PFS was by investigator (INV) assessment, with a supplemental analysis of PFS by blinded independent central review (BICR).

METHODS

Patients received veliparib or placebo with carboplatin-paclitaxel (6 cycles) and as maintenance (30 additional cycles). The primary analysis compared PFS in the veliparib-throughout arm to the carboplatin-paclitaxel only arm in the BRCA mutation (BRCAm), homologous recombination deficiency (HRD), and intention-to-treat (ITT) populations. Exploratory analyses of PFS in BRCA wildtype (BRCAwt), homologous recombination proficient (HRP), and HRD + BRCAwt populations were also performed. PFS per BICR and overall concordance rates between INV and BICR assessments were analyzed.

RESULTS

Hazard ratios for PFS by INV and BICR were consistent in each of the primary analysis and exploratory populations. In the ITT population, median PFS per INV was 23.5 months in the veliparib-throughout arm versus 17.3 months in the control arm (hazard ratio [HR] 0.683, 95% confidence interval [CI] 0.562-0.831; P < 0.001). Median PFS by BICR was 29.3 months versus 19.2 months (HR 0.687, 95% CI 0.504-0.806). In the ITT population, the overall concordance rates between INV and BICR were 78% and 75% for the veliparib-throughout and control arms, respectively.

CONCLUSIONS

Hazard ratios for PFS per BICR and per INV were consistent, with no suggestion of investigator bias. These findings support the reliability of PFS by INV in ovarian cancer trials.

Local Investigators Significantly Overestimate Overall Response Rates Compared to Blinded Independent Central Reviews in Phase 2 Oncology Trials

The overall response rate (ORR) is a largely adopted outcome measure in early-phase oncology trials. ORR is highly relevant in cancer drug development at the time of deciding whether to move to confirmatory phase 3 trials; moreover, ORR is gaining increasing relevance in fast-track registration procedures. No systematic analysis has been conducted so far to investigate whether a discrepancy exists between ORR assessed by local investigators and those assessed by blinded reviewers in phase 2 oncology trials. In this study, we carried out a search in the clinicaltrials.gov and EudraCT databases, looking at the trials reporting the results of both investigator-assessed and independently-assessed ORR. A discrepancy index was obtained by calculating the ratio of each investigator-assessed ORR on the corresponding independently assessed ORR, so that a discrepancy index >1 indicates that the investigator was "more optimistic," whereas a discrepancy index <1 indicates the opposite. We also analyzed different subgroups (by tumor type, by drug type, by year). Twenty trials met the search criteria; in some cases, >1 comparison was conducted in the trial, so that the total number of comparisons analyzed was 33. The estimated mean discrepancy index was 1.175 (95% confidence interval, 1.083-1.264; n = 33). In conclusion, local investigators significantly overestimate ORR compared to paired blinded reviewers in phase 2 oncology trials. This may represent a risk in drug development, when deciding whether to move to confirmatory, more expensive phase 3 trials. Blinded independent central review should be used in ORR assessment if a more conservative estimate of treatment efficacy is required, as in the case of fast-track drug developments leading to accelerated approvals of cancer therapies.