A simulation model of colorectal cancer surveillance and recurrence

A Review on Calibration Methods of Cancer Simulation Models

Calibration, a critical step in the development of simulation models, involves adjusting unobservable parameters to ensure that the outcomes of the model closely align with observed target data. This process is particularly vital in cancer simulation models with a natural history component where direct data to inform natural history parameters are rarely available. This work reviews the literature of cancer simulation models with a natural history component and identifies the calibration approaches used in these models with respect to the following attributes: calibration target, goodness-of-fit (GOF) measure, parameter search algorithm, acceptance criteria, and stopping rules. After a comprehensive search of the PubMed database from 1981 to June 2023, 68 studies were included in the review. Nearly all (n=66) articles specified the calibration targets, and most articles (n=56) specified the parameter search algorithms they used, whereas goodness-of-fit metric (n=51) and acceptance criteria/stopping rule (n=45) were reported for fewer times. The most frequently used calibration targets were incidence, mortality, and prevalence, whose data sources primarily come from cancer registries and observational studies. The most used goodness-of-fit measure was weighted mean squared error. Random search has been the predominant method for parameter search, followed by grid search and Nelder-mead method. Machine learning-based algorithms, despite their fast advancement in the recent decade, has been underutilized in the cancer simulation models. More research is needed to compare different parameter search algorithms used for calibration.

Key points

This work reviewed the literature of cancer simulation models with a natural history component and identified the calibration approaches used in these models with respect to the following attributes: calibration target, goodness-of-fit (GOF) measure, parameter search algorithm, acceptance criteria, and stopping rules.Random search has been the predominant method for parameter search, followed by grid search and Nelder-mead method.Machine learning-based algorithms, despite their fast advancement in the recent decade, has been underutilized in the cancer simulation models. Furthermore, more research is needed to compare different parameter search algorithms used for calibration.

Using Machine Learning to Leverage Biomarker Change and Predict Colorectal Cancer Recurrence

PURPOSE

The risk of colorectal cancer (CRC) recurrence after primary treatment varies across individuals and over time. Using patients' most up-to-date information, including carcinoembryonic antigen (CEA) biomarker profiles, to predict risk could improve personalized decision making.

METHODS

We used electronic health record data from an integrated health system on a cohort of patients diagnosed with American Joint Committee on Cancer stage I-III CRC between 2008 and 2013 (N = 3,970) and monitored until recurrence or end of follow-up. We addressed missingness in recurrence outcomes and longitudinal CEA measures, and engineered CEA features using current and past biomarker values for inclusion in a risk prediction model. We used a discrete time Superlearner model to evaluate various algorithms for predicting recurrence. We evaluated the time-varying discrimination and calibration of the algorithms and assessed the role of individual predictors.

RESULTS

Recurrence was documented in 448 (11.3%) patients. XGBoost with depth = 1 (XGB-D1) predicted recurrence substantially better than all other algorithms at all time points, with AUC ranging from 0.87 (95% CI, 0.86 to 0.88) at 6 months to 0.94 (95% CI, 0.92 to 0.96) at 54 months. The only variable used by XGB-D1 was 6-month change in log CEA. Predicted 1-year risk of recurrence was nearly zero for patients whose log CEA did not increase in the last 6 months, between 12.2% and 34.1% for patients whose log CEA increased between 0.10 and 0.40, and 43.6% for those with a log CEA increase >0.40. Compared with XGB, penalized regression approaches (lasso, ridge, and elastic net) performed poorly, with AUCs ranging from 0.58 to 0.69.

CONCLUSION

A flexible, machine learning approach that incorporated longitudinal CEA information yielded a simple and high-performing model for predicting recurrence on the basis of 6-month change in log CEA.

Cost-Effectiveness of Surveillance after Metastasectomy of Stage IV Colorectal Cancer

Surveillance of stage IV colorectal cancer (CRC) after curative-intent metastasectomy can be effective for detecting asymptomatic recurrence. Guidelines for various forms of surveillance exist but are supported by limited evidence. We aimed to determine the most cost-effective strategy for surveillance following curative-intent metastasectomy of stage IV CRC. We performed a decision analysis to compare four active surveillance strategies involving clinic visits and investigations elicited from National Comprehensive Cancer Network (NCCN) recommendations. Markov model inputs included data from a population-based cohort and literature-derived costs, utilities, and probabilities. The primary outcomes were costs (2021 Canadian dollars) and quality-adjusted life years (QALYs) gained. Over a 10-year base-case time horizon, surveillance with follow-ups every 12 months for 5 years was most economically favourable at a willingness-to-pay threshold of CAD 50,000 per QALY. These patterns were generally robust in the sensitivity analysis. A more intensive surveillance strategy was only favourable with a much higher willingness-to-pay threshold of approximately CAD 425,000 per QALY, with follow-ups every 3 months for 2 years then every 12 months for 3 additional years. Our findings are consistent with NCCN guidelines and justify the need for additional research to determine the impact of surveillance on CRC outcomes.

A two-phase approach to re-calibrating expensive computer simulation for sex-specific colorectal neoplasia development modeling

BACKGROUND

Medical evidence from more recent observational studies may significantly alter our understanding of disease incidence and progression, and would require recalibration of existing computational and predictive disease models. However, it is often challenging to perform recalibration when there are a large number of model parameters to be estimated. Moreover, comparing the fitting performances of candidate parameter designs can be difficult due to significant variation in simulated outcomes under limited computational budget and long runtime, even for one simulation replication.

METHODS

We developed a two-phase recalibration procedure. As a proof-of-the-concept study, we verified the procedure in the context of sex-specific colorectal neoplasia development. We considered two individual-based state-transition stochastic simulation models, estimating model parameters that govern colorectal adenoma occurrence and its growth through three preclinical states: non-advanced precancerous polyp, advanced precancerous polyp, and cancerous polyp. For the calibration, we used a weighted-sum-squared error between three prevalence values reported in the literature and the corresponding simulation outcomes. In phase 1 of the calibration procedure, we first extracted the baseline parameter design from relevant studies on the same model. We then performed sampling-based searches within a proper range around the baseline design to identify the initial set of good candidate designs. In phase 2, we performed local search (e.g., the Nelder-Mead algorithm), starting from the candidate designs identified at the end of phase 1. Further, we investigated the efficiency of exploring dimensions of the parameter space sequentially based on our prior knowledge of the system dynamics.

RESULTS

The efficiency of our two-phase re-calibration procedure was first investigated with CMOST, a relatively inexpensive computational model. It was then further verified with the V/NCS model, which is much more expensive. Overall, our two-phase procedure showed a better goodness-of-fit than the straightforward employment of the Nelder-Mead algorithm, when only a limited number of simulation replications were allowed. In addition, in phase 2, performing local search along parameter space dimensions sequentially was more efficient than performing the search over all dimensions concurrently.

CONCLUSION

The proposed two-phase re-calibration procedure is efficient at estimating parameters of computationally expensive stochastic dynamic disease models.

Predicting Survival for Patients with Malignant Pleural Effusion: Development of the CONCH Prognostic Model

BACKGROUND

Malignant pleural effusion (MPE) is a frequent complication of advanced malignancies that leads to a poor quality of life and limits treatment options.

OBJECTIVE

The objective of this study was to identify biomarkers of survival in patients with MPE, which will greatly facilitate the clinical management of this complication.

METHODS

This retrospective study recruited patients who had been pathologically diagnosed with MPE, regardless of the type of primary cancer, at Beijing Chao-Yang Hospital over 158 months. Demographic, clinical, hematological, and pleural fluid data were collected and analyzed as potential predictors of survival, and a new predictive model was developed based on Cox and logistic regression analyses.

RESULTS

In our alternative prognostic model (n = 281), four routinely detected variables, namely, carcinoembryonic antigen (CEA) level, monocyte count, N-terminal pro B-type natriuretic peptide (NT-pro-BNP) level, and pleural effusion chloride level on admission, were identified as predictors (the CONCH prognostic score). Patients were divided into three prognosis subgroups based on risk stratification, with median survival periods of 17, 11, and 5 months, respectively. In comparison with the low-risk group, patients in the medium- and high-risk groups showed significantly poorer survival (medium-risk group: hazard ratio [HR], 1.586; 95% confidence interval [CI], 1.047-2.402; P = 0.029; high-risk group: HR, 4.389; 95% CI, 2.432-7.921; P < 0.001).

CONCLUSION

Four routinely detected variables were used to develop the CONCH scoring system, which was confirmed to be an accurate prognostic score for patients with MPE. This system can guide the selection of interventions and management for MPE.

A novel nanomissile targeting two biomarkers and accurately bombing CTCs with doxorubicin

Rare circulating tumor cells (CTCs) cause >50% of primary colorectal cancer survivors to develop deadly metastasis at 3-5 years after surgery; the current chemotherapies can do nothing about these cells. Herein, we synthesized a novel doxorubicin (DOX)-entrapped mesoporous silica nanoparticle (MSN), covalently-conjugated with two aptamers, for simultaneously targeting EpCAM and CD44, the typical surface biomarkers of colorectal CTCs. The nanomissile can specifically capture the metastasis-prone CTCs spiked in healthy human blood in a competitive-binding manner. The binding not only accurately delivers DOX into the cancer cells via the biomarker-mediated endocytosis to inhibit CTC viability through the DOX-dependent mechanism, but also inhibits the adhesion of cancer cells to the endothelium and the consequent transmembrane migration through the DOX-independent mechanism. The molecular entity of the conjugate and its pharmaceutical DOX encapsulation-releasing capacity are well-demonstrated via various physiochemical characterizations including gel electrophoresis, which proves the >8-hour biostability of the nanomissile in blood, long enough for it to chase CTCs in mice and synergistically inhibit the CTC-induced lung metastasis more potently than its single aptamer-conjugated counterparts and DOX itself. The present strategy may pave a new avenue for safe and effective cancer metastasis chemoprevention.

Tailored Treatment of Colorectal Cancer: Surgical, Molecular, and Genetic Considerations

Colorectal cancer (CRC) is a complex cancer disease, and approximately 40% of the surgically cured patients will experience cancer recurrence within 5 years. During recent years, research has shown that CRC treatment should be tailored to the individual patient due to the wide variety of risk factors, genetic factors, and surgical complexity. In this review, we provide an overview of the considerations that are needed to provide an individualized, patient-tailored treatment. We emphasize the need to assess the predictors of CRC, and we summarize the latest research on CRC genetics and immunotherapy. Finally, we provide a summary of the significant variations in the colon and rectal anatomy that is important to consider in an individualized surgical approach. For the individual patient with CRC, a tailored treatment approach is needed in the preoperative, operative, and postoperative phase.

The Melanoma MAICare Framework: A Microsimulation Model for the Assessment of Individualized Cancer Care

Recently, new but expensive treatments have become available for metastatic melanoma. These improve survival, but in view of the limited funds available, cost-effectiveness needs to be evaluated. Most cancer cost-effectiveness models are based on the observed clinical events such as recurrence- free and overall survival. Times at which events are recorded depend not only on the effectiveness of treatment but also on the timing of examinations and the types of tests performed. Our objective was to construct a microsimulation model framework that describes the melanoma disease process using a description of underlying tumor growth as well as its interaction with diagnostics, treatments, and surveillance. The framework should allow for exploration of the impact of simultaneously altering curative treatment approaches in different phases of the disease as well as altering diagnostics. The developed framework consists of two components, namely, the disease model and the clinical management module. The disease model consists of a tumor level, describing growth and metastasis of the tumor, and a patient level, describing clinically observed states, such as recurrence and death. The clinical management module consists of the care patients receive. This module interacts with the disease process, influencing the rate of transition between tumor growth states at the tumor level and the rate of detecting a recurrence at the patient level. We describe the framework as the required input and the model output. Furthermore, we illustrate model calibration using registry data and data from the literature.

Using Active Learning for Speeding up Calibration in Simulation Models

BACKGROUND

Most cancer simulation models include unobservable parameters that determine disease onset and tumor growth. These parameters play an important role in matching key outcomes such as cancer incidence and mortality, and their values are typically estimated via a lengthy calibration procedure, which involves evaluating a large number of combinations of parameter values via simulation. The objective of this study is to demonstrate how machine learning approaches can be used to accelerate the calibration process by reducing the number of parameter combinations that are actually evaluated.

METHODS

Active learning is a popular machine learning method that enables a learning algorithm such as artificial neural networks to interactively choose which parameter combinations to evaluate. We developed an active learning algorithm to expedite the calibration process. Our algorithm determines the parameter combinations that are more likely to produce desired outputs and therefore reduces the number of simulation runs performed during calibration. We demonstrate our method using the previously developed University of Wisconsin breast cancer simulation model (UWBCS).

RESULTS

In a recent study, calibration of the UWBCS required the evaluation of 378 000 input parameter combinations to build a race-specific model, and only 69 of these combinations produced results that closely matched observed data. By using the active learning algorithm in conjunction with standard calibration methods, we identify all 69 parameter combinations by evaluating only 5620 of the 378 000 combinations.

CONCLUSION

Machine learning methods hold potential in guiding model developers in the selection of more promising parameter combinations and hence speeding up the calibration process. Applying our machine learning algorithm to one model shows that evaluating only 1.49% of all parameter combinations would be sufficient for the calibration.