Evaluation of Prognostic and Predictive Models in the Oncology Clinic

Use of Patient-Reported Outcomes in Risk Prediction Model Development to Support Cancer Care Delivery: A Scoping Review

PURPOSE

The integration of patient-reported outcomes (PROs) into electronic health records (EHRs) has enabled systematic collection of symptom data to manage post-treatment symptoms. The use and integration of PRO data into routine care are associated with overall treatment success, adherence, and satisfaction. Clinical trials have demonstrated the prognostic value of PROs including physical function and global health status in predicting survival. It is unknown to what extent routinely collected PRO data are used in the development of risk prediction models (RPMs) in oncology care. The objective of the scoping review is to assess how PROs are used to train risk RPMs to predict patient outcomes in oncology care.

METHODS

Using the scoping review methodology outlined in the Joanna Briggs Institute Manual for Evidence Synthesis, we searched four databases (MEDLINE, CINAHL, Embase, and Web of Science) to locate peer-reviewed oncology articles that used PROs as predictors to train models. Study characteristics including settings, clinical outcomes, and model training, testing, validation, and performance data were extracted for analyses.

RESULTS

Of the 1,254 studies identified, 18 met inclusion criteria. Most studies performed retrospective analyses of prospectively collected PRO data to build prediction models. Post-treatment survival was the most common outcome predicted. Discriminative performance of models trained using PROs was better than models trained without PROs. Most studies did not report model calibration.

CONCLUSION

Systematic collection of PROs in routine practice provides an opportunity to use patient-reported data to develop RPMs. Model performance improves when PROs are used in combination with other comprehensive data sources.

Treatment Response Biomarkers: Working Toward Personalized Radiotherapy for Lung Cancer

Owing to major advances in the field of radiation oncology, patients with lung cancer can now receive technically individualized radiotherapy treatments. Nevertheless, in the era of precision oncology, radiotherapy-based treatment selection needs to be improved as many patients do not benefit or are not offered optimum therapies. Cost-effective robust biomarkers can address this knowledge gap and lead to individuals being offered more bespoke treatments leading to improved outcome. This narrative review discusses some of the current achievements and challenges in the realization of personalized radiotherapy delivery in patients with lung cancer.

Fixing the Leaky Pipe: How to Improve the Uptake of Patient-Reported Outcomes-Based Prognostic and Predictive Models in Cancer Clinical Practice

PURPOSE

This discussion paper outlines challenges and proposes solutions for successfully implementing prediction models that incorporate patient-reported outcomes (PROs) in cancer practice.

METHODS

We organized a full-day multidisciplinary meeting of people with expertise in cancer care delivery, PRO collection, PRO use in prediction modeling, computing, implementation, and decision science. The discussions presented here focused on identifying challenges to the development, implementation and use of prediction models incorporating PROs, and suggesting possible solutions.

RESULTS

Specific challenges and solutions were identified across three broad areas. (1) Understanding decision making and implementation: necessitating multidisciplinary collaboration in the early stages and throughout; early stakeholder engagement to define the decision problem and ensure acceptability of PROs in prediction; understanding patient/clinician interpretation of PRO predictions and uncertainty to optimize prediction impact; striving for model integration into existing electronic health records; and early regulatory alignment. (2) Recognizing the limitations to PRO collection and their impact on prediction: incorporating validated, clinically important PROs to maximize model generalizability and clinical engagement; and minimizing missing PRO data (resulting from both structural digital exclusion and time-varying factors) to avoid exacerbating existing inequalities. (3) Statistical and modeling challenges: incorporating statistical methods to address missing data; ensuring predictive modeling recognizes complex causal relationships; and considering temporal and geographic recalibration so that model predictions reflect the relevant population.

CONCLUSION

Developing and implementing PRO-based prediction models in cancer care requires extensive multidisciplinary working from the earliest stages, recognition of implementation challenges because of PRO collection and model presentation, and robust statistical methods to manage missing data, causality, and calibration. Prediction models incorporating PROs should be viewed as complex interventions, with their development and impact assessment carried out to reflect this.

Decision aids to assist patients and professionals in choosing the right treatment for kidney failure

Background

Kidney services vary in the way they involve people with kidney failure (PwKF) in treatment decisions as management needs change. We discuss how decision-science applications support proactively PwKF to make informed decisions between treatment options with kidney professionals.

Methods

A conceptual review of findings about decision making and use of decision aids in kidney services, synthesized with reference to: the Making Informed Decisions-Individually and Together (MIND-IT) multiple stakeholder decision makers framework; and the Medical Research Council-Complex Intervention Development and Evaluation research framework.

Results

This schema represents the different types of decision aids that support PwKF and professional reasoning as they manage kidney disease individually and together; adjustments at micro, meso and macro levels supports integration in practice.

Conclusion

Innovating services to meet clinical guidelines on enhancing shared decision making processes means enabling all stakeholders to use decision aids to meet their goals within kidney pathways at individual, service and organizational levels.

Validation of PREDICT Version 2.2 in a Retrospective Cohort of Indian Women With Operable Breast Cancer

PURPOSE

Online prediction models that use known prognostic factors in breast cancer (BC) are routinely used to assist in decisions for adjuvant therapy. PREDICT Version 2.2 (P2.2) is one such online tool, which uses tumor size, lymph node involvement, grade, age, hormone receptor status, human epidermal growth factor receptor 2 (HER2) status, and Ki67. We performed an external validation in a retrospective cohort of patients treated at a tertiary center in India.

METHODS

Women with operable BC between 2008 and 2016 with nonmetastatic, T1-T2 invasive, and HER2 receptor-negative BC and with available 5-year overall survival (OS) data were selected. Median predicted 5-year OS rates were used to calculate predicted events for the whole cohort and subgroups. The chi-square test was used to evaluate the goodness of fit of the tool.

RESULTS

Of 11,760 cases registered between 2008 and 2016, 2,783 (23.66%) eligible patients with a median age of 50 (26-70) years and a median pT size of 2.5 (0.1-5) cm, 2,037 (73.19%) with grade 3 tumors, 1,172 (42.11%) with node-positive disease, 817 (29.35%) with triple-negative breast cancer, and 1,966 (70.64%) with HR-positive BC were included in the analysis. The observed 5-year OS and predicted 5-year OS in the whole cohort were 94.8% and 90.00%, respectively, with an absolute difference of 4.8% (95% CI, 3.417 to 6.198, P < .001). The observed 5-year OS and predicted 5-year OS were also different in various subgroups.

CONCLUSION

PREDICT version 2.2 overestimated the number of deaths, with lower predicted 5-year OS compared with the observed value, in this retrospective Indian cohort. The reasons for this discrepancy could be differing biologic characteristics and possible selection bias in our cohort. We recommend a prospective validation of PREDICT in Indian patients and advocate caution in its use until such validation is achieved.

Outcome prediction models incorporating clinical variables for Head and Neck Squamous cell Carcinoma: A systematic review of methodological conduct and risk of bias

Multiple outcome prediction models have been developed for Head and Neck Squamous Cell Carcinoma (HNSCC). This systematic review aimed to identify HNSCC outcome prediction model studies, assess their methodological quality and identify those with potential utility for clinical practice. Inclusion criteria were mucosal HNSCC prognostic prediction model studies (development or validation) incorporating clinically available variables accessible at time of treatment decision making and predicting tumour-related outcomes. Eligible publications were identified from PubMed and Embase. Methodological quality and risk of bias were assessed using the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) and prediction model risk of bias assessment tool (PROBAST). Eligible publications were categorised by study type for reporting. 64 eligible publications were identified; 55 reported model development, 37 external validations, with 28 reporting both. CHARMS checklist items relating to participants, predictors, outcomes, handling of missing data, and some model development and evaluation procedures were generally well-reported. Less well-reported were measures accounting for model overfitting and model performance measures, especially model calibration. Full model information was poorly reported (3/55 model developments), specifically model intercept, baseline survival or full model code. Most publications (54/55 model developments, 28/37 external validations) were found to have high risk of bias, predominantly due to methodological issues in the PROBAST analysis domain. The identified methodological issues may affect prediction model accuracy in heterogeneous populations. Independent external validation studies in the local population and demonstration of clinical impact are essential for the clinical implementation of outcome prediction models.

Open Source Repository and Online Calculator of Prediction Models for Diagnosis and Prognosis in Oncology

(1) Background: The main aim was to develop a prototype application that would serve as an open-source repository for a curated subset of predictive and prognostic models regarding oncology, and provide a user-friendly interface for the included models to allow online calculation. The focus of the application is on providing physicians and health professionals with patient-specific information regarding treatment plans, survival rates, and side effects for different expected treatments. (2) Methods: The primarily used models were the ones developed by our research group in the past. This selection was completed by a number of models, addressing the same cancer types but focusing on other outcomes that were selected based on a literature search in PubMed and Medline databases. All selected models were publicly available and had been validated TRIPOD (Transparent Reporting of studies on prediction models for Individual Prognosis Or Diagnosis) type 3 or 2b. (3) Results: The open source repository currently incorporates 18 models from different research groups, evaluated on datasets from different countries. Model types included logistic regression, Cox regression, and recursive partition analysis (decision trees). (4) Conclusions: An application was developed to enable physicians to complement their clinical judgment with user-friendly patient-specific predictions using models that have received internal/external validation. Additionally, this platform enables researchers to display their work, enhancing the use and exposure of their models.

Prognostic value of patient-reported outcome measures (PROMs) in adults with non-small cell Lung Cancer: a scoping review

BACKGROUND

There is growing interest in the collection and use of patient-reported outcome measures (PROMs) to support clinical decision making in patients with non-small cell lung cancer (NSCLC). However, an overview of research into the prognostic value of PROMs is currently lacking.

AIM

To explore to what extent, how, and how robustly the value of PROMs for prognostic prediction has been investigated in adults diagnosed with NSCLC.

METHODS

We systematically searched Medline, Embase, CINAHL Plus and Scopus for English-language articles published from 2011 to 2021 that report prognostic factor study, prognostic model development or validation study. Example data charting forms from the Cochrane Prognosis Methods Group guided our data charting on study characteristics, PROMs as predictors, predicted outcomes, and statistical methods. Two reviewers independently charted the data and critically appraised studies using the QUality In Prognosis Studies (QUIPS) tool for prognostic factor studies, and the risk of bias assessment section of the Prediction model Risk Of Bias ASsessment Tool (PROBAST) for prognostic model studies.

RESULTS

Our search yielded 2,769 unique titles of which we included 31 studies, reporting the results of 33 unique analyses and models. Out of the 17 PROMs used for prediction, the EORTC QLQ-C30 was most frequently used (16/33); 12/33 analyses used PROM subdomain scores instead of the overall scores. PROMs data was mostly collected at baseline (24/33) and predominantly used to predict survival (32/33) but seldom other clinical outcomes (1/33). Almost all prognostic factor studies (26/27) had moderate to high risk of bias and all four prognostic model development studies had high risk of bias.

CONCLUSION

There is an emerging body of research into the value of PROMs as a prognostic factor for survival in people with NSCLC but the methodological quality of this research is poor with significant bias. This warrants more robust studies into the prognostic value of PROMs, in particular for predicting outcomes other than survival. This will enable further development of PROM-based prediction models to support clinical decision making in NSCLC.