Can oncologists predict survival for patients with progressive disease after standard chemotherapies?

Integrating Artificial Intelligence and Machine Learning Into Cancer Clinical Trials

The practice of oncology requires analyzing and synthesizing abundant data. From the patient's workup to determine eligibility to the therapies received to the post-treatment surveillance, practitioners must constantly juggle, evaluate, and weigh decision-making based on their best understanding of information at hand. These complex, multifactorial decisions have a tremendous opportunity to benefit from data-driven machine learning (ML) methods to drive opportunities in artificial intelligence (AI). Within the past 5 years, we have seen AI move from simply a promising opportunity to being used in prospective trials. Here, we review recent efforts of AI in clinical trials that have moved the needle towards improved prediction of actionable outcomes, such as predicting acute care visits, short term mortality, and pathologic extranodal extension. We then pause and reflect on how these AI models ask a different question than traditional statistics models that readers may be more familiar with; how then should readers conceptualize and interpret AI models that they are not as familiar with. We end with what we believe are promising future opportunities for AI in oncology, with an eye towards allowing the data to inform us through unsupervised learning and generative models, rather than asking AI to perform specific functions.

Clinicians' Prediction of Survival and Prognostic Confidence in Patients with Advanced Cancer in Three East Asian Countries

Background: Little is known about accuracy and confidence of clinicians' prediction of survival (CPS) in East-Asian countries. Objective: We aimed to examine accuracy of CPS for 7-, 21-, and 42-day survival in palliative inpatients and its association with prognostic confidence. Design: An international prospective cohort study in Japan (JP), Korea (KR), and Taiwan (TW). Setting/Subjects: Subjects were inpatients with advanced cancer admitted to 37 palliative care units in three countries. Measurements: Discrimination of CPS was investigated through sensitivity, specificity, overall accuracy, and area under the receiver operating characteristics curves (AUROCs) according to 7-, 21-, and 42-day survival. The accuracies of CPS were compared with those of Performance Status-based Palliative Prognostic Index (PS-PPI). Clinicians were instructed to rate confidence level on a 0-10-point scale. Results: A total of 2571 patients were analyzed. The specificity was highest at 93.2-100.0% for the 7-day CPS, and sensitivity was highest at 71.5-86.8% for the 42-day CPS. The AUROCs of the seven-day CPS were 0.88, 0.94, and 0.89, while those of PS-PPI were 0.77, 0.69, and 0.69 for JP, KR, and TW, respectively. As for 42-day prediction, sensitivities of PS-PPI were higher than those of CPS. Clinicians' confidence was strongly associated with the accuracy of prediction in all three countries (all p-values <0.01). Conclusions: CPS accuracies were highest (0.88-0.94) for the seven-day survival prediction. CPS was more accurate than PS-PPI in all timeframe prediction except 42-day prediction in KR. Prognostic confidence was significantly associated with the accuracy of CPS.

Two case reports using a proposed oral risk assessment tool for older people near the end of life

Objectives

We developed a prototype technique that expresses the need for intervention and the effectiveness of the treatment when “not being at risk of injury to the oral cavity or to general health” due to the presence of teeth or prostheses is taken as the desired outcome of dental treatment for older people near the end of life. The objective of this study was to use the prototype risk assessment matrix to identify the risk for each patient according to their course and show the effectiveness of treatment.

Material and Methods

We produced a prototype Dental Risk Map (Dental R‐map) based on the risk map method of risk management. Risk is classified into three levels according to the level of tolerability: (A) Risk for which watchful waiting should be included among measures to be considered; (B) risk for which intervention should be considered; or (C) risk requiring urgent intervention.

Results

We report the application of this technique to two men in their 80s. Both were assessed as risk tolerability Level C, requiring immediate intervention. Dental treatment eliminated this risk in one and reduced it to Level B in the other.

Conclusions

We developed the prototype Dental R‐map to identify oral risks and indicate the need for intervention to address these risks and the effectiveness of treatment for older people near the end of life. We used the Dental R‐map for two patients and successfully avoided oral risks that might cause physical injury in both cases until their deaths.

How well do neurosurgeons predict survival in patients with high-grade glioma?

Due to the lack of reliable prognostic tools, prognostication and surgical decisions largely rely on the neurosurgeons’ clinical prediction skills. The aim of this study was to assess the accuracy of neurosurgeons’ prediction of survival in patients with high-grade glioma and explore factors possibly associated with accurate predictions. In a prospective single-center study, 199 patients who underwent surgery for high-grade glioma were included. After surgery, the operating surgeon predicted the patient’s survival using an ordinal prediction scale. A survival curve was used to visualize actual survival in groups based on this scale, and the accuracy of clinical prediction was assessed by comparing predicted and actual survival. To investigate factors possibly associated with accurate estimation, a binary logistic regression analysis was performed. The surgeons were able to differentiate between patients with different lengths of survival, and median survival fell within the predicted range in all groups with predicted survival < 24 months. In the group with predicted survival > 24 months, median survival was shorter than predicted. The overall accuracy of surgeons’ survival estimates was 41%, and over- and underestimations were done in 34% and 26%, respectively. Consultants were 3.4 times more likely to accurately predict survival compared to residents (p = 0.006). Our findings demonstrate that although especially experienced neurosurgeons have rather good predictive abilities when estimating survival in patients with high-grade glioma on the group level, they often miss on the individual level. Future prognostic tools should aim to beat the presented clinical prediction skills.

Is early integration of palliative home care in oncology treatment feasible and acceptable for advanced cancer patients and their health care providers? A phase 2 mixed-methods study

BACKGROUND

To support the early integration of palliative home care (PHC) in cancer treatment, we developed the EPHECT intervention and pilot tested it with 30 advanced cancer patients in Belgium using a pre post design with no control group. We aim to determine the feasibility, acceptability and perceived effectiveness of the EPHECT intervention.

METHODS

Interviews with patients (n = 16 of which 11 dyadic with family caregivers), oncologists and GPs (n = 11) and a focus group with the PHC team. We further analyzed the study materials and logbooks of the PHC team (n = 8). Preliminary effectiveness was assessed with questionnaires EORTC QLQ C-30, HADS and FAMCARE and were filled in at baseline and 12, 18 and 24 weeks.

RESULTS

In the interviews after the intervention period, patients reported feelings of safety and control and an optimized quality of life. The PHC team could focus on more than symptom management because they were introduced earlier in the trajectory of the patient. Telephone-based contact appeared to be insufficient to support interprofessional collaboration. Furthermore, some family caregivers reported that the nurse of the PHC team was focused little on them.

CONCLUSION

Nurses of PHC teams are able to deliver early palliative care to advanced cancer patients. However, more attention needs to be given to family caregivers as caregiver and client. Furthermore, the home visits by the PHC team have to be further evaluated and adapted. Lastly, professionals have to find a more efficient way to discuss future care.

Estimation of Risk of Recurrence and Toxicity Among Oncologists and Patients With Resected Breast Cancer: A Quantitative Study

Shared decision-making regarding adjuvant systemic therapy in breast cancer is based on both properly conveying information about the prognosis of the disease and the benefits and risks of adjuvant treatment, as well as the patient's ability to understand this information. This work proposed to analyze oncologists' and patients' perceptions of the risk of recurrence with and without chemotherapy and toxicity, and the factors influencing said impressions. This was a prospective, cross-sectional, multicenter study that involved 281 breast cancer patients and 23 oncologists. Prognosis (risk of recurrence with and without chemotherapy and risk of severe toxicity with chemotherapy) and shared decision making (SDM) questionnaires were completed by all participants; breast cancer patients also filled out the 18-item Brief Symptom Inventory (BSI-18). Oncologists' prediction of risk of relapse without and with chemotherapy (30.4 and 13.3%) and risk of severe toxicity (9.8%) were more optimistic than those of breast cancer patients (78.6, 29.6, and 61%, respectively). The greater the severity, the higher the risk of relapse according to the oncologists (p = 0.001); not so for the patients. Older physicians and more experienced ones predicted lower risk of relapse with and without chemotherapy and less severe toxicity than younger doctors and those with less experience (p < 0.001). Oncologists' SDM and their prediction of risk of relapsing with chemotherapy correlated negatively with patients' SDM and their prediction of risk of severe toxicity (p < 0.01). There is a positive correlation between psychological distress (BSI-18) and prognosis of risk of recurrence with chemotherapy in breast cancer patients (p < 0.001). These results stress the importance of improving doctor-patient communication in SDM. In breast cancer patients undergoing treatment with curative intent, expectations of being cured would increase and treatment-related anxiety would decrease by enhancing doctor-patient communication to coincide more with respect to risk of relapse and toxicity, thereby enhancing patients' quality of life.

Prognostic performance of proteomic testing in advanced non-small cell lung cancer: a systematic literature review and meta-analysis

OBJECTIVE

Timely assessment of patient-specific prognosis is critical to oncology care involving a shared decision-making approach, but clinical prognostic factors traditionally used in NSCLC have limitations. We examine a proteomic test to address these limitations.

METHODS

This study examines the prognostic performance of the VeriStrat blood-based proteomic test that measures the inflammatory disease state of patients with advanced NSCLC. A systematic literature review (SLR) was performed, yielding cohorts in which the hazard ratio (HR) was reported for overall survival (OS) of patients with VeriStrat Poor (VSPoor) test results versus VeriStrat Good (VSGood). A study-level meta-analysis of OS HRs was performed in subgroups defined by lines of therapy and treatment regimens.

RESULTS

Twenty-four cohorts met SLR criteria. Meta-analyses in five subgroups (first-line platinum-based chemotherapy, second-line single-agent chemotherapy, first-line EGFR-tyrosine kinase inhibitor (TKI) therapy, and second- and higher-line TKI therapy, and best supportive care) resulted in statistically significant (p ≤ .001) summary effect sizes for OS HRs of 0.42, 0.54, 0.41, 0.52, and 0.50, respectively, indicating increased OS by about two-fold for patients who test VSGood. No significant heterogeneity was seen in any subgroup (p > .05).

CONCLUSIONS

Advanced NSCLC patients classified VSGood have significantly longer OS than those classified VSPoor. The summary effect size for OS HRs around 0.4-0.5 indicates that the expected median survival of those with a VSGood classification is approximately 2-2.5 times as long as those with VSPoor. The robust prognostic performance of the VeriStrat test across various lines of therapy and treatment regimens has clinical implications for treatment shared decision-making and potential for novel treatment strategies.

How palliative care professionals deal with predicting life expectancy at the end of life: predictors and accuracy

PURPOSE

To assess the accuracy of hospice staff in predicting survival of subjects admitted to hospice, exploring the factors considered most helpful by the hospice staff to accurately predict survival.

METHODS

Five physicians and 11 nurses were asked to predict survival at admission of 827 patients. Actual and predicted survival times were divided into ≤ 1 week, 2-3 weeks, 4-8 weeks, and ≥ 2 months and the accuracy of the estimates was calculated. The staff members were each asked to score 17 clinical variables that guided them in predicting survival and we analyzed how these variables impacted the accuracy.

RESULTS

Physicians' and nurses' accuracy of survival of the patients was 46% and 40% respectively. Survival was underestimated in 20% and 12% and overestimated in 34% and 48% of subjects. Both physicians and nurses considered metastases, comorbidities, dyspnea, disability, tumor site, neurological symptoms, and confusion very important in predicting patients' survival with nurses assigning more importance to intestinal symptoms and pain too. All these factors, with the addition of cough and/or bronchial secretions, were associated with physicians' greater accuracy. In the multivariable models, intestinal symptoms and confusion continued to be associated with greater predictive accuracy. No factors appreciably raised nurses' accuracy.

CONCLUSIONS

Some clinical symptoms rated as relevant by the hospice staff could be important for predicting survival. However, only intestinal symptoms and confusion significantly improved the accuracy of physicians' predictions, despite the high prevalence of overestimated survival.

Accuracy and Prognostic Significance of Oncologists' Estimates and Scenarios for Survival Time in Advanced Gastric Cancer

BACKGROUND

Worst-case, typical, and best-case scenarios for survival, based on simple multiples of an individual's expected survival time (EST), estimated by their oncologist, are a useful way of formulating and explaining prognosis. We aimed to determine the accuracy and prognostic significance of oncologists' estimates of EST, and the accuracy of the resulting scenarios for survival time, in advanced gastric cancer.

MATERIALS AND METHODS

Sixty-six oncologists estimated the EST at baseline for each of the 152 participants they enrolled in the INTEGRATE trial. We hypothesized that oncologists' estimates of EST would be unbiased (∼50% would be longer or shorter than the observed survival time [OST]); imprecise (<33% within 0.67-1.33 times the OST); independently predictive of overall survival (OS); and accurate at deriving scenarios for survival time with approximately 10% of patients dying within a quarter of their EST (worst-case scenario), 50% living within half to double their EST (typical scenario), and 10% living three or more times their EST (best-case scenario).

RESULTS

Oncologists' estimates of EST were unbiased (45% were shorter than the OST, 55% were longer); imprecise (29% were within 0.67-1.33 times observed); moderately discriminative (Harrell's C-statistic 0.62, p = .001); and an independently significant predictor of OS (hazard ratio, 0.89; 95% confidence interval, 0.83-0.95; p = .001) in a Cox model including performance status, number of metastatic sites, neutrophil-to-lymphocyte ratio ≥3, treatment group, age, and health-related quality of life (EORTC-QLQC30 physical function score). Scenarios for survival time derived from oncologists' estimates were remarkably accurate: 9% of patients died within a quarter of their EST, 57% lived within half to double their EST, and 12% lived three times their EST or longer.

CONCLUSION

Oncologists' estimates of EST were unbiased, imprecise, moderately discriminative, and independently significant predictors of OS. Simple multiples of the EST accurately estimated worst-case, typical, and best-case scenarios for survival time in advanced gastric cancer.

IMPLICATIONS FOR PRACTICE

Results of this study demonstrate that oncologists' estimates of expected survival time for their patients with advanced gastric cancer were unbiased, imprecise, moderately discriminative, and independently significant predictors of overall survival. Simple multiples of the expected survival time accurately estimated worst-case, typical, and best-case scenarios for survival time in advanced gastric cancer.

Impact of Accuracy of Survival Predictions on Quality of End-of-Life Care Among Patients With Metastatic Cancer Who Receive Radiation Therapy

PURPOSE:

For patients treated with palliative radiation, we examined the association between life expectancy predictions by radiation oncologists and aggressive end-of-life care.

MATERIALS AND METHODS:

We included decedents from a study that assessed the ability of oncologists to predict survival of patients with metastatic cancer who received radiation. We identified patients who died within 12 months of study enrollment to assess accuracy of predictions. Aggressive end-of-life care was defined by the National Quality Forum, ASCO Quality Oncology Practice Initiative metrics, and advanced radiation modalities in the last month of life. Survival predictions were categorized as follows: correct (< 12 months), 12 to 18 months, 18 to 24 months, and more than 24 months. We assessed association between prediction and aggressive end-of-life care using a generalized estimation equation.

RESULTS:

Of 489 decedents, we identified 467 encounters with survival estimates. Overall, 156 decedents (32%) met at least one metric of aggressive end-of-life care. Factors associated with aggressive end-of-life care included younger age, female sex, primary cancer diagnosis, no brain metastases, and private insurance. In each encounter when an oncologist predicted survival, 363 predictions (78%) were correct (< 12 months), 54 (11%) incorrectly predicted 12 to 18 months, 27 (6%) predicted 18 to 24 months, and 23 (5%) predicted more than 24 months. Compared with patients who had encounters that had correct survival predictions, patients predicted to live more than 24 months were more likely to meet at least one metric of aggressive end-of-life care (odds ratio, 2.55; 95% CI, 1.09 to 5.99; P = .03).

CONCLUSION:

Inaccurate survival predictions by oncologists are associated with more aggressive end-of-life care for patients with advanced cancer.

Survivability Prediction of Colorectal Cancer Patients: A System with Evolving Features for Continuous Improvement

Prediction in health care is closely related with the decision-making process. On the one hand, accurate survivability prediction can help physicians decide between palliative care or other practice for a patient. On the other hand, the notion of remaining lifetime can be an incentive for patients to live a fuller and more fulfilling life. This work presents a pipeline for the development of survivability prediction models and a system that provides survivability predictions for years one to five after the treatment of patients with colon or rectal cancer. The functionalities of the system are made available through a tool that balances the number of necessary inputs and prediction performance. It is mobile-friendly and facilitates the access of health care professionals to an instrument capable of enriching their practice and improving outcomes. The performance of survivability models was compared with other existing works in the literature and found to be an improvement over the current state of the art. The underlying system is capable of recalculating its prediction models upon the addition of new data, continuously evolving as time passes.

Impact of a multivariate serum-based proteomic test on physician treatment recommendations for advanced non-small-cell lung cancer

OBJECTIVE

The VeriStrat ¹ (VS) test is intended to help guide treatment decisions for patients with advanced non-small-cell lung cancer (NSCLC) without an EGFR-sensitizing mutation, classifying patients into two categories. Patients classified as VSGood have a favorable prognosis and significant clinical response to EGFR tyrosine kinase inhibitors (TKIs). Patients classified as VSPoor have a less favorable prognosis and exhibit no significant response to EGFR-TKIs. The objective of this paper is to assess the real-world impact of VS test results on physicians' treatment recommendations including referrals for best supportive care (BSC).

METHODS

Between 1 January 2012 and 1 November 2016, physician respondents were asked to complete standardized questionnaires before and after receiving VS results in patients meeting criteria for the intended use of the VS test. This study evaluated three endpoints: whether physicians followed VS test results in making treatment recommendations, the extent to which tests results changed these treatment recommendations, and the patterns of care subsequent to VS testing.

RESULTS

Of the tests ordered by 989 physicians, 2494 VS tests had completed treatment recommendation questionnaires both prior to and after testing. Prior to VS testing, physicians were considering treatment with EGFR-TKIs for 2250 patients (90%). The VS test classified 1950 patients as VSGood and 544 patients as VSPoor. For patients classified as VSPoor, physicians recommended BSC for 25% of patients and standard systemic treatments such as chemotherapies for 65% of patients. Consistent with previous publications, physicians recommended EGFR-TKI therapy for only 10% of VSPoor patients but for 89% of VSGood patients. Overall, physician's treatment recommendations were consistent with test results in 98% of cases. Availability of test results decreased ineffective treatment recommendations by 89% for VSPoor patients.

CONCLUSIONS

Among physicians ordering VS, the test significantly influenced treatment recommendations for patients with NSCLC, reducing ineffective and expensive treatment at the end of life.

A Systematic Review of Predictions of Survival in Palliative Care: How Accurate Are Clinicians and Who Are the Experts?

Background

Prognostic accuracy in palliative care is valued by patients, carers, and healthcare professionals. Previous reviews suggest clinicians are inaccurate at survival estimates, but have only reported the accuracy of estimates on patients with a cancer diagnosis.

Objectives

To examine the accuracy of clinicians’ estimates of survival and to determine if any clinical profession is better at doing so than another.

Data Sources

MEDLINE, Embase, CINAHL, and the Cochrane Database of Systematic Reviews and Trials. All databases were searched from the start of the database up to June 2015. Reference lists of eligible articles were also checked.

Eligibility Criteria

Inclusion criteria: patients over 18, palliative population and setting, quantifiable estimate based on real patients, full publication written in English. Exclusion criteria: if the estimate was following an intervention, such as surgery, or the patient was artificially ventilated or in intensive care.

Study Appraisal and Synthesis Methods

A quality assessment was completed with the QUIPS tool. Data on the reported accuracy of estimates and information about the clinicians were extracted. Studies were grouped by type of estimate: categorical (the clinician had a predetermined list of outcomes to choose from), continuous (open-ended estimate), or probabilistic (likelihood of surviving a particular time frame).

Results

4,642 records were identified; 42 studies fully met the review criteria. Wide variation was shown with categorical estimates (range 23% to 78%) and continuous estimates ranged between an underestimate of 86 days to an overestimate of 93 days. The four papers which used probabilistic estimates tended to show greater accuracy (c-statistics of 0.74–0.78). Information available about the clinicians providing the estimates was limited. Overall, there was no clear “expert” subgroup of clinicians identified.

Limitations

High heterogeneity limited the analyses possible and prevented an overall accuracy being reported. Data were extracted using a standardised tool, by one reviewer, which could have introduced bias. Devising search terms for prognostic studies is challenging. Every attempt was made to devise search terms that were sufficiently sensitive to detect all prognostic studies; however, it remains possible that some studies were not identified.

Conclusion

Studies of prognostic accuracy in palliative care are heterogeneous, but the evidence suggests that clinicians’ predictions are frequently inaccurate. No sub-group of clinicians was consistently shown to be more accurate than any other.

Implications of Key Findings

Further research is needed to understand how clinical predictions are formulated and how their accuracy can be improved.

Development of a model to predict breast cancer survival using data from the National Cancer Data Base

BACKGROUND

With the large amounts of data on patient, tumor, and treatment factors available to clinicians, it has become critically important to harness this information to guide clinicians in discussing a patient's prognosis. However, no widely accepted survival calculator is available that uses national data and includes multiple prognostic factors. Our objective was to develop a model for predicting survival among patients diagnosed with breast cancer using the National Cancer Data Base (NCDB) to serve as a prototype for the Commission on Cancer's "Cancer Survival Prognostic Calculator."

PATIENTS AND METHODS

A retrospective cohort of patients diagnosed with breast cancer (2003-2006) in the NCDB was included. A multivariable Cox proportional hazards regression model to predict overall survival was developed. Model discrimination by 10-fold internal cross-validation and calibration was assessed.

RESULTS

There were 296,284 patients for model development and internal validation. The c-index for the 10-fold cross-validation ranged from 0.779 to 0.788 after inclusion of all available pertinent prognostic factors. A plot of the observed versus predicted 5 year overall survival showed minimal deviation from the reference line.

CONCLUSION

This breast cancer survival prognostic model to be used as a prototype for building the Commission on Cancer's "Cancer Survival Prognostic Calculator" will offer patients and clinicians an objective opportunity to estimate personalized long-term survival based on patient demographic characteristics, tumor factors, and treatment delivered.