Validating Risk Prediction Models for Multiple Primaries and Competing Cancer Outcomes in Families With Li-Fraumeni Syndrome Using Clinically Ascertained Data

PURPOSE

There exists a barrier between developing and disseminating risk prediction models in clinical settings. We hypothesize that this barrier may be lifted by demonstrating the utility of these models using incomplete data that are collected in real clinical sessions, as compared with the commonly used research cohorts that are meticulously collected.

MATERIALS AND METHODS

Genetic counselors (GCs) collect family history when patients (ie, probands) come to MD Anderson Cancer Center for risk assessment of Li-Fraumeni syndrome, a genetic disorder characterized by deleterious germline mutations in the TP53 gene. Our clinical counseling-based (CCB) cohort consists of 3,297 individuals across 124 families (522 cases of single primary cancer and 125 cases of multiple primary cancers). We applied our software suite LFSPRO to make risk predictions and assessed performance in discrimination using AUC and in calibration using observed/expected (O/E) ratio.

RESULTS

For prediction of deleterious TP53 mutations, we achieved an AUC of 0.78 (95% CI, 0.71 to 0.85) and an O/E ratio of 1.66 (95% CI, 1.53 to 1.80). Using the LFSPRO.MPC model to predict the onset of the second cancer, we obtained an AUC of 0.70 (95% CI, 0.58 to 0.82). Using the LFSPRO.CS model to predict the onset of different cancer types as the first primary, we achieved AUCs between 0.70 and 0.83 for sarcoma, breast cancer, or other cancers combined.

CONCLUSION

We describe a study that fills in the critical gap in knowledge for the utility of risk prediction models. Using a CCB cohort, our previously validated models have demonstrated good performance and outperformed the standard clinical criteria. Our study suggests that better risk counseling may be achieved by GCs using these already-developed mathematical models.

LFSPROShiny: An Interactive R/Shiny App for Prediction and Visualization of Cancer Risks in Families With Deleterious Germline TP53 Mutations

PURPOSE

LFSPRO is an R library that implements risk prediction models for Li-Fraumeni syndrome (LFS), a genetic disorder characterized by deleterious germline mutations in the TP53 gene. To facilitate the use of these models in clinics, we developed LFSPROShiny, an interactive R/Shiny interface of LFSPRO that allows genetic counselors (GCs) to perform risk predictions without any programming components and further visualize the risk profiles of their patients to aid the decision-making process.

METHODS

LFSPROShiny implements two models that have been validated on multiple LFS patient cohorts: a competing risk model that predicts cancer-specific risks for the first primary and a recurrent-event model that predicts the risk of a second primary tumor. Starting with a visualization template, we keep regular contact with GCs, who ran LFSPROShiny in their counseling sessions, to collect feedback and discuss potential improvement. On receiving the family history as input, LFSPROShiny renders the family into a pedigree and displays the risk estimates of the family members in a tabular format. The software offers interactive overlaid side-by-side bar charts for visualization of the patients' cancer risks relative to the general population.

RESULTS

We walk through a detailed example to illustrate how GCs can run LFSPROShiny in clinics from data preparation to downstream analyses and interpretation of results with an emphasis on the utilities that LFSPROShiny provides to aid decision making.

CONCLUSION

Since December 2021, we have applied LFSPROShiny to over 100 families from counseling sessions at the MD Anderson Cancer Center. Our study suggests that software tools with easy-to-use interfaces are crucial for the dissemination of risk prediction models in clinical settings, hence serving as a guideline for future development of similar models.

Validating risk prediction models for multiple primaries and competing cancer outcomes in families with Li-Fraumeni syndrome using clinically ascertained data at a single institute

Purpose

There exists a barrier between developing and disseminating risk prediction models in clinical settings. We hypothesize this barrier may be lifted by demonstrating the utility of these models using incomplete data that are collected in real clinical sessions, as compared to the commonly used research cohorts that are meticulously collected.

Patients and methods

Genetic counselors (GCs) collect family history when patients (i.e., probands) come to MD Anderson Cancer Center for risk assessment of Li-Fraumeni syndrome, a genetic disorder characterized by deleterious germline mutations in the TP53 gene. Our clinical counseling-based (CCB) cohort consists of 3,297 individuals across 124 families (522 cases of single primary cancer and 125 cases of multiple primary cancers). We applied our software suite LFSPRO to make risk predictions and assessed performance in discrimination using area under the curve (AUC), and in calibration using observed/expected (O/E) ratio.

Results

For prediction of deleterious TP53 mutations, we achieved an AUC of 0.81 (95% CI, 0.70 - 0.91) and an O/E ratio of 0.96 (95% CI, 0.70 - 1.21). Using the LFSPRO.MPC model to predict the onset of the second cancer, we obtained an AUC of 0.70 (95% CI, 0.58 - 0.82). Using the LFSPRO.CS model to predict the onset of different cancer types as the first primary, we achieved AUCs between 0.70 and 0.83 for sarcoma, breast cancer, or other cancers combined.

Conclusion

We describe a study that fills in the critical gap in knowledge for the utility of risk prediction models. Using a CCB cohort, our previously validated models have demonstrated good performance and outperformed the standard clinical criteria. Our study suggests better risk counseling may be achieved by GCs using these already-developed mathematical models.

LFSPROShiny: an interactive R/Shiny app for prediction and visualization of cancer risks in families with deleterious germline TP53 mutations

Purpose

LFSPRO is an R library that implements risk prediction models for Li-Fraumeni syndrome (LFS), a genetic disorder characterized by deleterious germline mutations in the TP53 gene. To facilitate the use of these models in clinics, we developed LFSPROShiny, an interactive R/Shiny interface of LFSPRO that allows genetic counselors (GCs) to perform risk predictions without any programming components, and further visualize the risk profiles of their patients to aid the decision-making process.

Methods

LFSPROShiny implements two models that have been validated on multiple LFS patient cohorts: a competing-risk model that predicts cancer-specific risks for the first primary, and a recurrent-event model that predicts the risk of a second primary tumor. Starting with a visualization template, we keep regular contact with GCs, who ran LFSPROShiny in their counseling sessions, to collect feedback and discuss potential improvement. Upon receiving the family history as input, LFSPROShiny renders the family into a pedigree, and displays the risk estimates of the family members in a tabular format. The software offers interactive overlaid side-by-side bar charts for visualization of the patients' cancer risks relative to the general population.

Results

We walk through a detailed example to illustrate how GCs can run LFSPROShiny in clinics, from data preparation to downstream analyses and interpretation of results with an emphasis on the utilities that LFSPROShiny provides to aid decision making.

Conclusion

Since Dec 2021, we have applied LFSPROShiny to over 100 families from counseling sessions at MD Anderson Cancer Center. Our study suggests that software tools with easy-to-use interfaces are crucial for the dissemination of risk prediction models in clinical settings, hence serving as a guideline for future development of similar models.