Use and misuse of waterfall plots

Efficacy assessment in phase I clinical trials: endpoints and challenges

The scope of phase I clinical trials in oncology goes beyond the conventional safety evaluation-only objectives of these trials in other specialties. Rather, most first-in-human oncology clinical trials have therapeutic intent, and efficacy signals observed in phase I trials can drive a go/no-go decision of advancing a new molecule to phase II testing. The complexity of efficacy assessment in the context of a small, heterogeneous patient population and a complex study design requires a more liberal perspective compared with later trial phases when looking into efficacy endpoints. Classically, in later-phase clinical trials, these endpoints would include the objective response rate, progression-free survival, and overall survival. However, new, evolving endpoints may be worth investigating when looking into the antitumor activity signals in phase I trials. Integration of all these endpoints into trial designs can improve the assessment of therapeutic efficacy during early drug development and guide decisions related to the further advancement of novel molecules into later phases. In this review, we discuss the advantages and pitfalls of different classic efficacy endpoints when evaluated as part of phase I trials in oncology and describe how challenges in assessing the antitumor activity of new drugs can be overcome through the incorporation of novel endpoints that have thus far proven successful in clinical trials.

Assessing immunotherapy response: going beyond RECIST by integrating early tumor growth kinetics

Objective

Assess the contribution of early tumor growth dynamics modeling to predict clinical outcomes in non-small cell lung cancer patients receiving immunotherapy, alongside standard RECIST 1.1 criteria.

Methods

Our retrospective studies used data from 861 patients with advanced NSCLC enrolled in three randomized Phase III trials evaluating immunotherapy plus chemotherapy were analyzed. Tumor size measurements up to two follow-up time points were used to fit a novel Gompertz model and estimate growth rate (GR) and kinetic parameters representing depth of response (A), speed of response (B), and long-term modulation (M). Correlations between these early tumor growth parameters and clinical outcomes such as progression-free survival (PFS) and time to response (TTR) were assessed. Descriptive and discriminative analyses were performed to delineate tumor growth dynamics across various response profiles based on RECIST 1.1 criteria.

Results

The novel Gompertz model accurately described early tumor growth kinetics in 861 non-small cell lung cancer patients treated with immunotherapy. Lower growth rate (GR) and model parameter M were associated with longer progression-free survival (PFS) (HR=0.897 and 7.47x10^-7, respectively). Higher GR and parameter A correlated with shorter time to response (HR=0.575 and 0.696, respectively). Responders had significantly lower A (p=1.51e-53) and higher GR (p=0.4e-12) than non-responders. Non-durable stable disease patients had higher GR (p=0.0001) and parameter B (p=0.0002) compared to late responders. Early tumor growth parameters showed potential for predicting long-term outcomes and treatment response patterns.

Correlation between hematological parameters and PET/CT metabolic parameters in patients with head and neck cancer

Background

Systemic inflammation is predictive of the overall survival in cancer patients and is related to the density of immune cells in the tumor microenvironment of cancer, which in turn correlates with ¹⁸F -fluorodeoxyglucose (FDG)-positron emission tomography/computed tomography (PET/CT) metabolic parameters (MPs). The density of tumor-infiltrating lymphocytes (TILs) in the microenvironment has the potential to be a biomarker that can be used clinically to optimize patient selection in oropharyngeal head and neck squamous cell carcinoma (HNSCC). There is little to no data regarding the association of systemic inflammation with PET/CT-MPs, especially in HNSCC. This study aimed to evaluate the correlation between markers of host inflammation, namely blood neutrophil-to-lymphocyte ratio (NLR) and platelet-to-lymphocyte ratio (PLR), with the PET/CT-MPs standardized uptake value (SUV), metabolic tumor volume (MTV), and total lesion glycolysis (TLG) of the primary tumor, derived from FDG-PET/CT in patients with nonmetastatic (cM0) HNSCC before treatment. We hypothesized that NLR and PLR at baseline are positively correlated with PET/CT-MPs.

Methods

A retrospective review of consecutive patients with HNSCC with a pretreatment PET/CT was performed. NLR and PLR were computed using complete blood counts measured within 10 days before the start of any treatment. The correlation between NLR and PLR with PET/CT-MPs was evaluated with Spearman's rho test.

Results

Seventy-one patients were analyzed. Overall survival (OS) at 1, 2, and 3 years was 86%, 76%, and 68%. PLR was found to be correlated with MTV (rho = 0.26, P = .03) and TLG (rho = 0.28, P = .02) but not with maximum SUV or mean SUV. There was no correlation between NLR and the analyzed PET/CT-MPs. TLG was associated with worse survival in uni- and multivariable analysis, but no other PET/CT-MPs were associated with either OS or disease-specific survival (DSS). NLR and PLR were associated with OS and DSS on uni- and multivariable analysis.

Conclusions

In patients with HNSCC before any treatment such as definitive radio (chemo)therapy or oncologic surgery followed by adjuvant RT, baseline PLR correlated with MTV and TLG but not with SUV. NLR was not correlated with any PET/CT-MPs analyzed in our study. Confirmatory studies are needed, and a potential interaction between tumor microenvironment, host inflammation, and FDG-PET/CT measures warrants further investigation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13014-022-02112-4.

Prognostic value of preoperative soluble interleukin 2 receptor α as a novel immune biomarker in epithelial ovarian cancer

PURPOSE

Epithelial ovarian cancer (EOC) is regarded as the deadliest gynecological cancer, and the demand for novel noninvasive prognostic biomarkers remains significant. This study aimed to investigate the prognostic value of preoperative blood biomarkers in EOC patients.

METHODS

In total, 73 patients who had undergone ovarian mass resection were enrolled. Serum concentration of biomarkers, including soluble interleukin 2 receptor α (sIL-2R), was measured 1-2 weeks before surgery. Independent prognostic factors for progression-free survival (PFS) were investigated with multivariate Cox regression analysis. A prognostic model was subsequently developed and evaluated by discrimination, calibration and clinical net benefit. Furthermore, transcriptome data of 376 EOC cases from The Cancer Genome Atlas (TCGA) were analyzed with ESTIMATE, CIBERSORT and Maftools algorithm to evaluate the correlation of IL2RA expression with tumor immune microenvironment and immunotherapeutic response.

RESULTS

High sIL-2R concentration was found to be the only significant prognostic blood biomarker for PFS by multivariate Cox regression analysis in our center. A prognostic nomogram was developed with satisfactory discrimination, calibration and clinical net benefit. In addition, higher IL2RA expression was significantly associated with higher immune scores, activated CD4⁺ T cells, M2 macrophages and resting dendritic cells in TCGA data. Furthermore, IL2RA expression was closely related to TMB scores.

CONCLUSIONS

sIL-2R is a potential prognostic immune biomarker for EOC patients, and a comprehensive prognostic model comprising sIL-2R with satisfactory discrimination and clinical appliance was developed. Therefore, we recommend routine sIL-2R testing in EOC patients.

Pre-pembrolizumab neutrophil-to-lymphocyte ratio (NLR) predicts the efficacy of second-line pembrolizumab treatment in urothelial cancer regardless of the pre-chemo NLR

Neutrophil-to-lymphocyte ratio (NLR) was reported to be associated with prognosis of urothelial cancer (UC) patients receiving systemic chemotherapy or immunotherapy. However, it has not been elucidated how preceding first-line chemotherapy affects NLR and subsequent second-line pembrolizumab treatment. This multicenter study analyzed 458 patients with metastatic UC who received first-line chemotherapy and second-line pembrolizumab with regard to pre-chemotherapy and pre-pembrolizumab NLR in association with the efficacy of chemotherapy and pembrolizumab treatment. NLR was increased in 47% while decreased in 53% of patients before and after first-line chemotherapy. High pre-chemotherapy NLR (≥ 3) was significantly associated with unfavorable overall (OS, P = 0.0001) and progression-free (P < 0.0001) survivals after first-line chemotherapy. However, pre-chemotherapy NLR showed only modest influence on radiological response and survival after second-line pembrolizumab treatment, whereas pre-pembrolizumab NLR showed higher association. NLR decrease was associated with partial response or greater objective response by first-line chemotherapy, while NLR increase was associated with higher patient age. In conclusion, immediate pre-chemotherapy and pre-pembrolizumab NLR was significantly associated with efficacy of the following treatment, respectively. However, even patients with high pre-chemotherapy NLR achieved favorable OS if they had their NLR reduced by chemotherapy, whereas those with high pre-chemotherapy NLR yielded unfavorable OS if they had their NLR remained high after chemotherapy, suggesting that chemotherapy may have differential effect on the efficacy of subsequent pembrolizumab treatment in UC patients.

Tumor morphology and location associate with immune cell composition in pleomorphic sarcoma

Background

Soft-tissue sarcomas (STS) are a rare group of mesenchymal malignancies that account for approximately 1% of adult human cancer. Undifferentiated pleomorphic sarcoma (UPS) is one of the most common subtypes of adult STS. Clinical stratification of UPS patients has not evolved for decades and continues to rely on tumor-centric metrics including tumor size and depth. Our understanding of how the tumor microenvironment correlates to these clinicopathologic parameters remains limited.

Methods

Here, we performed single-cell flow cytometric immune-based profiling of 15 freshly resected UPS tumors and integrated this analysis with clinical, histopathologic, and outcomes data using both a prospective and retrospective cohort of UPS patients.

Results

We uncovered a correlation between physiologic and anatomic properties of UPS tumors and the composition of immune cells in the tumor microenvironment. Specifically, we identified an inverse correlation between tumor-infiltrating CD8 + T cells and UPS tumor size; and a positive correlation between tumor-infiltrating CD8 + T cells and overall survival. Moreover, we demonstrate an association between anatomical location (deep or superficial) and frequency of CD4 + PD1^hi infiltrating T cells in UPS tumors.

Conclusions

Our study provides an immune-based analysis of the tumor microenvironment in UPS patients and describes the different composition of tumor infiltrating lymphocytes based on size and tumor depth.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00262-021-02935-2.

Motor deficits at presentation and predictors of overall survival in central nervous system lymphomas

PURPOSE

Central nervous system lymphomas (CNSL) can present with motor and non-motor symptoms. In many central nervous system tumors, motor deficits are associated with significant morbidity and functional impairment, and correlate with worse prognosis. CNSLs however, often exhibit remarkable response to chemotherapy and radiotherapy with corresponding symptom improvement. We investigate the survival outcomes and trajectories of motor and functional recovery in a cohort of patients presenting with and without initial motor deficits.

METHODS

Patients who underwent biopsy and with a histologically confirmed CNSL between 2008 and 2019 were retrospectively identified. Baseline demographic variables, comorbidities, presenting symptoms, histological type, neuroimaging features (location and number of lesions), and treatment administered (pre- and post-operative steroid use and chemotherapy regime) were recorded. Dates of death were obtained from the National Registry of Births and Deaths. Motor power and performance status at admission, 1 month and 6 months were determined.

RESULTS

We identified 119 patients, of whom 34% presented with focal motor deficits. The median overall survival (OS) was 26.6 months. Those with focal motor deficits had longer OS (median 42.4 months) than those without (median 23.3 months; p = 0.047). In multivariate Cox analysis, age (HR 1.04 per year; p = 0.003), CCI (HR 1.31 per point; p < 0.001), leptomeningeal/ependymal involvement (HR 2.53; p = 0.016), thalamus involvement (HR 0.34; p = 0.019), neutrophil:lymphocyte ratio (HR 1.06 per point; p = 0.034), positive HIV status (HR 5.31; p = 0.003), preoperative steroids use (HR 0.49; p = 0.018), postoperative high-dose steroids (HR 0.26; p < 0.001) and postoperative low-dose steroids (HR 0.28; p = 0.010) were significant predictors of OS. By one month, 43% of surviving patients had full power, increasing to 61% by six months.

CONCLUSION

A significant proportion of patients with initial motor deficits recovered in motor strength by six months. In our population, those presenting with motor deficits had paradoxically better overall survival.

Objective response rate assessment in oncology: Current situation and future expectations

The tumor objective response rate (ORR) is an important parameter to demonstrate the efficacy of a treatment in oncology. The ORR is valuable for clinical decision making in routine practice and a significant end-point for reporting the results of clinical trials. World Health Organization and Response Evaluation Criteria in Solid Tumors (RECIST) are anatomic response criteria developed mainly for cytotoxic chemotherapy. These criteria are based on the visual assessment of tumor size in morphological images provided by computed tomography (CT) or magnetic resonance imaging. Anatomic response criteria may not be optimal for biologic agents, some disease sites, and some regional therapies. Consequently, modifications of RECIST, Choi criteria and Morphologic response criteria were developed based on the concept of the evaluation of viable tumors. Despite its limitations, RECIST v1.1 is validated in prospective studies, is widely accepted by regulatory agencies and has recently shown good performance for targeted cancer agents. Finally, some alternatives of RECIST were developed as immune-specific response criteria for checkpoint inhibitors. Immune RECIST criteria are based essentially on defining true progressive disease after a confirmatory imaging. Some graphical methods may be useful to show longitudinal change in the tumor burden over time. Tumor tissue is a tridimensional heterogenous mass, and tumor shrinkage is not always symmetrical; thus, metabolic response assessments using positron emission tomography (PET) or PET/CT may reflect the viability of cancer cells or functional changes evolving after anticancer treatments. The metabolic response can show the benefit of a treatment earlier than anatomic shrinkage, possibly preventing delays in drug approval. Computer-assisted automated volumetric assessments, quantitative multimodality imaging in radiology, new tracers in nuclear medicine and finally artificial intelligence have great potential in future evaluations.

Quantitative Imaging Assessment for Clinical Trials in Oncology

BACKGROUND

Objective radiographic assessment is crucial for accurately evaluating therapeutic efficacy and patient outcomes in oncology clinical trials. Imaging assessment workflow can be complex; can vary with institution; may burden medical oncologists, who are often inadequately trained in radiology and response criteria; and can lead to high interobserver variability and investigator bias. This article reviews the development of a tumor response assessment core (TRAC) at a comprehensive cancer center with the goal of providing standardized, objective, unbiased tumor imaging assessments, and highlights the web-based platform and overall workflow. In addition, quantitative response assessments by the medical oncologists, radiologist, and TRAC are compared in a retrospective cohort of patients to determine concordance.

PATIENTS AND METHODS

The TRAC workflow includes an image analyst who pre-reviews scans before review with a board-certified radiologist and then manually uploads annotated data on the proprietary TRAC web portal. Patients previously enrolled in 10 lung cancer clinical trials between January 2005 and December 2015 were identified, and the prospectively collected quantitative response assessments by the medical oncologists were compared with retrospective analysis of the same dataset by a radiologist and TRAC.

RESULTS

This study enlisted 49 consecutive patients (53% female) with a median age of 60 years (range, 29-78 years); 2 patients did not meet study criteria and were excluded. A linearly weighted kappa test for concordance for TRAC versus radiologist was substantial at 0.65 (95% CI, 0.46-0.85; standard error [SE], 0.10). The kappa value was moderate at 0.42 (95% CI, 0.20-0.64; SE, 0.11) for TRAC versus oncologists and only fair at 0.34 (95% CI, 0.12-0.55; SE, 0.11) for oncologists versus radiologist.

CONCLUSIONS

Medical oncologists burdened with the task of tumor measurements in patients on clinical trials may introduce significant variability and investigator bias, with the potential to affect therapeutic response and clinical trial outcomes. Institutional imaging cores may help bridge the gap by providing unbiased and reproducible measurements and enable a leaner workflow.

Acute kidney injury as an independent predictor of infection and malignancy: the NARA-AKI cohort study

BACKGROUND

Acute kidney injury (AKI) is associated with higher mortality and cardiovascular events. However, association between AKI and non-cardiac events such as infection or malignancy is largely unknown.

METHODS

This is a retrospective cohort study. Inclusion criteria were adults who underwent non-cardiac surgery from 2007 to 2011 at Nara Medical University Hospital. Exclusion criteria were urological surgery, obstetric surgery, missing creatinine values peri-operatively, and pre-operative dialysis. The end of observation period was at the end of 2015 or loss to follow-up. A predictor was AKI defined by KDIGO criteria within 1-week post-operatively. Outcomes were hospitalization for infection or diagnoses of malignancy. Associations between AKI and outcomes were examined by Cox regression models.

RESULTS

Among 6692 subjects, 445 (6.6%) developed AKI. During median follow-up of 4.0 years, there were 485 hospitalizations for infection and 1138 diagnoses of malignancy (2.0 and 5.1 events/100 patient-years, respectively). After adjustment for potential confounders, AKI was independently associated with hospitalization for infection and diagnoses of malignancy (Hazard ratio [95% confidence interval]: 1.64 [1.23-2.20] and 1.31 [1.06-1.61], respectively). Excluding recurrence of malignancy from outcomes and analyses limited to those who recover renal function by the time of discharge yielded similar results. Absolute lymphocyte counts were significantly lower and neutrophil-to-lymphocyte ratios were significantly higher among those with AKI.

CONCLUSIONS

AKI was significantly associated with hospitalization for infection and development of malignancy during long-term follow-up. Those with AKI might be in persistent immunosuppressed state.

Assessment of Accuracy of Waterfall Plot Representations of Response Rates in Cancer Treatment Published in Medical Journals

IMPORTANCE

Response rates are a well-recognized outcome of clinical trials and provide an objective measure of drug activity.

OBJECTIVES

To quantify the difference between objective response rate and visual representation of response in waterfall plots in recent articles in major medical journals and to assess the change in frequency over time with which waterfall plots are used.

DESIGN, SETTING, AND PARTICIPANTS

In a cross-sectional study, original articles of 6 top journals between July 2016 and June 2018 were manually reviewed to identify articles including a waterfall plot to describe a treatment effect of cancer therapy. Response rates visually represented in waterfall plots were compared with response rates reported as study outcomes. The number of original articles with a waterfall plot as a percentage of total original articles was evaluated, sampling articles from January, February, and March for the years 2004, 2008, 2012, 2016, and 2018.

MAIN OUTCOMES AND MEASURES

Difference between response rates depicted in waterfall plots and response rates reported as study outcomes.

RESULTS

One hundred twenty-six articles were selected for analysis. Of the 97 articles reporting investigator-assessed response rates, waterfall plots showed response rates a median (interquartile range) of 6.1% (1.8%-12.0%) higher than rates derived from investigator assessment. Forty-two articles reported response rates based on central assessment as an outcome, and waterfall plots showed response rates a median (interquartile range) of 12.0% (7.7%-18.5%) higher compared with centrally assessed response rates. The estimated percentage of original articles using waterfall plots increased from 0% in 2004 to 7% in 2018.

CONCLUSIONS AND RELEVANCE

This study suggests that waterfall plots are becoming more common in oncology literature. Waterfall plots provide a visual overestimate of response rate of cancer therapies and should be evaluated with caution.

Development of an algorithm for evaluating the impact of measurement variability on response categorization in oncology trials

BACKGROUND

Radiologic assessments of baseline and post-treatment tumor burden are subject to measurement variability, but the impact of this variability on the objective response rate (ORR) and progression rate in specific trials has been unpredictable on a practical level. In this study, we aimed to develop an algorithm for evaluating the quantitative impact of measurement variability on the ORR and progression rate.

METHODS

First, we devised a hierarchical model for estimating the distribution of measurement variability using a clinical trial dataset of computed tomography scans. Next, a simulation method was used to calculate the probability representing the effect of measurement errors on categorical diagnoses in various scenarios using the estimated distribution. Based on the probabilities derived from the simulation, we developed an algorithm to evaluate the reliability of an ORR (or progression rate) (i.e., the variation in the assessed rate) by generating a 95% central range of ORR (or progression rate) results if a reassessment was performed. Finally, we performed validation using an external dataset. In the validation of the estimated distribution of measurement variability, the coverage level was calculated as the proportion of the 95% central ranges of hypothetical second readings that covered the actual burden sizes. In the validation of the evaluation algorithm, for 100 resampled datasets, the coverage level was calculated as the proportion of the 95% central ranges of ORR results that covered the ORR from a real second assessment.

RESULTS

We built a web tool for implementing the algorithm (publicly available at http://studyanalysis2017.pythonanywhere.com/ ). In the validation of the estimated distribution and the algorithm, the coverage levels were 93 and 100%, respectively.

CONCLUSIONS

The validation exercise using an external dataset demonstrated the adequacy of the statistical model and the utility of the developed algorithm. Quantification of variation in the ORR and progression rate due to potential measurement variability is essential and will help inform decisions made on the basis of trial data.

From waterfall plots to spaghetti plots in early oncology clinical development

Waterfall plots are used to describe changes in tumor size observed in clinical studies. They are frequently used to illustrate the overall drug response in oncology clinical trials because of its simple representation of results. Unfortunately, this visual display suffers a number of limitations including (1) potential misguidance by masking the time dynamics of tumor size, (2) ambiguous labelling of the y-axis, and (3) low data-to-ink ratio. We offer some alternatives to address these shortcomings and recommend moving away from waterfall plots to the benefit of plots showing the individual time profiles of sum of lesion diameters (according to RECIST). The spider plot presents the individual changes in tumor measurements over time relative to baseline tumor burden. Baseline tumor size is a well-known confounding factor of drug effect which has to be accounted for when analyzing data in early clinical trials. While spider plots are conveniently correct for baseline tumor size, they cannot be presented in isolation. Indeed, percentage change from baseline has suboptimal statistical properties (including skewed distribution) and can be overly optimistic in favor of drug efficacy. We argued that plots of raw data (referred to as spaghetti plots) should always accompany spider plots to provide an equipoised illustration of the drug effect on lesion diameters.

ENABLE (Exportable Notation and Bookmark List Engine): an Interface to Manage Tumor Measurement Data from PACS to Cancer Databases

Oncologists evaluate therapeutic response in cancer trials based on tumor quantification following selected "target" lesions over time. At our cancer center, a majority of oncologists use Response Evaluation Criteria in Solid Tumors (RECIST) v1.1 quantifying tumor progression based on lesion measurements on imaging. Currently, our oncologists handwrite tumor measurements, followed by multiple manual data transfers; however, our Picture Archiving Communication System (PACS) (Carestream Health, Rochester, NY) has the ability to export tumor measurements, making it possible to manage tumor metadata digitally. We developed an interface, "Exportable Notation and Bookmark List Engine" (ENABLE), which produces prepopulated RECIST v1.1 worksheets and compiles cohort data and data models from PACS measurement data, thus eliminating handwriting and manual data transcription. We compared RECIST v1.1 data from eight patients (16 computed tomography exams) enrolled in an IRB-approved therapeutic trial with ENABLE outputs: 10 data fields with a total of 194 data points. All data in ENABLE's output matched with the existing data. Seven staff were taught how to use the interface with a 5-min explanatory instructional video. All were able to use ENABLE successfully without additional guidance. We additionally assessed 42 metastatic genitourinary cancer patients with available RECIST data within PACS to produce a best response waterfall plot. ENABLE manages tumor measurements and associated metadata exported from PACS, producing forms and data models compatible with cancer databases, obviating handwriting and the manual re-entry of data. Automation should reduce transcription errors and improve efficiency and the auditing process.

Timing and extent of response in colorectal cancer: critical review of current data and implication for future trials

The identification of new surrogate endpoints for advanced colorectal cancer is becoming crucial and, along with drug development, it represents a research field increasingly studied. Although overall survival (OS) remains the strongest trial endpoint available, it requires larger sample size and longer periods of time for an event to happen. Surrogate endpoints such as progression free survival (PFS) or response rate (RR) may overcome these issues but, as such, they need to be prospectively validated before replacing the real endpoints; moreover, they often bear many other limitations. In this narrative review we initially discuss the role of time-to-event endpoints, objective response and response rate as surrogates of OS in the advanced colorectal cancer setting, discussing also how such measures are influenced by the tumor assessment criteria currently employed. We then report recent data published about early tumor shrinkage and deepness of response, which have recently emerged as novel potential endpoint surrogates, discussing their strengths and weaknesses and providing a critical comment. Despite being very compelling, the role of such novel response measures is yet to be confirmed and their surrogacy with OS still needs to be further investigated within larger and well-designed trials.

Current and Evolving Methods to Visualize Biological Data in Cancer Research

Although the measurements of clinical outcomes for cancer treatments have become diverse and complex, there remains a need for clear, easily interpreted representations of patients' experiences. With oncology trials increasingly reporting non-time-to-event outcomes, data visualization has evolved to incorporate parameters such as responses to therapy, duration and degree of response, and novel representations of underlying tumor biology. We review both commonly used and newly developed methods to display outcomes in oncology, with a focus on those that have evolved to represent complex datasets.

Strength of Validation for Surrogate End Points Used in the US Food and Drug Administration's Approval of Oncology Drugs

OBJECTIVE

To determine the strength of the surrogate-survival correlation for cancer drug approvals based on a surrogate.

PARTICIPANTS AND METHODS

We performed a retrospective study of the US Food and Drug Administration (FDA) database, with focused searches of MEDLINE and Google Scholar. Among cancer drugs approved based on a surrogate end point, we examined previous publications assessing the strength of the surrogate-survival correlation. Specifically, we identified the percentage of surrogate approvals lacking any formal analysis of the strength of the surrogate-survival correlation, and when conducted, the strength of such correlations.

RESULTS

Between January 1, 2009, and December 31, 2014, the FDA approved marketing applications for 55 indications based on a surrogate, of which 25 were accelerated approvals and 30 were traditional approvals. We could not find any formal analyses of the strength of the surrogate-survival correlation in 14 out of 25 accelerated approvals (56%) and 11 out of 30 traditional approvals (37%). For accelerated approvals, just 4 approvals (16%) were made where a level 1 analysis (the most robust way to validate a surrogate) had been performed, with all 4 studies reporting low correlation (r≤0.7). For traditional approvals, a level 1 analysis had been performed for 15 approvals (50%): 8 (53%) reported low correlation (r≤0.7), 4 (27%) medium correlation (r>0.7 to r<0.85), and 3 (20%) high correlation (r≥0.85) with survival.

CONCLUSIONS

The use of surrogate end points for drug approval often lacks formal empirical verification of the strength of the surrogate-survival association.

Observer variability in RECIST-based tumour burden measurements: a meta-analysis

BACKGROUND

Response Evaluation Criteria in Solid Tumours (RECIST)-based tumour burden measurements involve observer variability, the extent of which ought to be determined.

METHODS

A literature search identified studies on observer variability during manual measurements of tumour burdens via computed tomography according to the RECIST guideline. The 95% limit of agreement (LOA) values of relative measurement difference (RMD) were pooled using a random-effects model.

RESULTS

Twelve studies were included. Pooled 95% LOAs of RMD in measuring unidimensional longest diameters of single lesions ranged from -22.1% (95% confidence interval [CI], -30.3% to -14.0%) to 25.4% (95% CI, 17.2% to 33.5%) between observers and -17.8% (95% CI, -23.6% to -11.9%) to 16.1% (95% CI, 10.1% to 21.8%) for a single observer. Pooled 95% LOAs of RMD in measuring the sum of multiple lesions ranged from -19.2% (95% CI, -23.7% to -14.9%) to 19.5% (95% CI, 15.2% to 23.9%) between observers, and -9.8% (95% CI, -19.0% to -0.3%) to 13.1% (95% CI, 3.6% to 22.6%) for a single observer. Pooled 95% LOA of RMD in calculating the interval change of tumour burden with a single lesion ranged from -31.3% (95% CI, -46.0% to -16.5%) to 30.3% (95% CI, 15.3% to 44.8%) between observers. Studies on calculating the interval change of tumour burden for a single observer or with multiple lesions were lacking.

CONCLUSION

Interobserver RMD in measuring single tumour burden and calculating its interval change may exceed the 20% cut-off for progression. Variability decreased when tumour burden was measured by a single observer or assessed by the sum of multiple lesions.