Performance of Statistical and Machine Learning Risk Prediction Models for Surveillance Benefits and Failures in Breast Cancer Survivors

BACKGROUND

Machine learning (ML) approaches facilitate risk prediction model development using high-dimensional predictors and higher-order interactions at the cost of model interpretability and transparency. We compared the relative predictive performance of statistical and ML models to guide modeling strategy selection for surveillance mammography outcomes in women with a personal history of breast cancer (PHBC).

METHODS

We cross-validated seven risk prediction models for two surveillance outcomes, failure (breast cancer within 12 months of a negative surveillance mammogram) and benefit (surveillance-detected breast cancer). We included 9,447 mammograms (495 failures, 1,414 benefits, and 7,538 nonevents) from years 1996 to 2017 using a 1:4 matched case-control samples of women with PHBC in the Breast Cancer Surveillance Consortium. We assessed model performance of conventional regression, regularized regressions (LASSO and elastic-net), and ML methods (random forests and gradient boosting machines) by evaluating their calibration and, among well-calibrated models, comparing the area under the receiver operating characteristic curve (AUC) and 95% confidence intervals (CI).

RESULTS

LASSO and elastic-net consistently provided well-calibrated predicted risks for surveillance failure and benefit. The AUCs of LASSO and elastic-net were both 0.63 (95% CI, 0.60-0.66) for surveillance failure and 0.66 (95% CI, 0.64-0.68) for surveillance benefit, the highest among well-calibrated models.

CONCLUSIONS

For predicting breast cancer surveillance mammography outcomes, regularized regression outperformed other modeling approaches and balanced the trade-off between model flexibility and interpretability.

IMPACT

Regularized regression may be preferred for developing risk prediction models in other contexts with rare outcomes, similar training sample sizes, and low-dimensional features.

Abstract P1-08-04: Withdrawn

This abstract was withdrawn by the authors.

Citation Format: Trabert B, Bauer DC, Brinton LA, Buist DS, Cauley JA, Dallal CM, Gierach GL, Falk RT, Hue TF, Lacey, Jr. JV, LaCroix AZ, Tice JA, Xu X. Withdrawn [abstract]. In: Proceedings of the 2018 San Antonio Breast Cancer Symposium; 2018 Dec 4-8; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2019;79(4 Suppl):Abstract nr P1-08-04.

Cancer Risk Awareness and Concern among Women with a Family History of Breast or Ovarian Cancer

Women with a documented deleterious mutation in BRCA1 or BRCA2 are at substantially elevated risk for ovarian cancer. To understand what percentage of women with high-risk family histories know their risk is elevated we surveyed 1,885 women with a high- or moderate-risk family history and no personal history of breast or ovarian cancer, and asked about their perceived risk of breast and ovarian cancer. Among high-risk women, fewer than 20% reported use of genetic counseling, and knowledge of elevated risk of ovarian cancer was low. Prior genetic counseling was associated with greater perceived risk for ovarian cancer. Results suggest that most high-risk women (>75%) do not know their risk for ovarian cancer. Identification of potentially high-risk women for referral to genetic counseling may improve informed ovarian cancer risk management.

Tissue sampling frequency and breast pathology diagnoses following mammography: Time trends and age group analysis from the Breast Cancer Surveillance Consortium (BCSC)

559

Background: Pathology diagnoses in a well-characterized population of women can be used to identify tissue sampling and diagnosis trends following mammography. Methods: Screening and diagnostic mammography, patient characteristics, and pathology reports from the BCSC performed from 1996-2008 were identified. Diagnosis was based on the most severe pathology interpretation in the same breast within 60 days of a post-mammogram tissue sample. Age, mammogram year and type, breast density, and family history of breast cancer were evaluated for associations with tissue sampling and most severe pathology diagnosis. Results: 4,022,506 mammograms (88.5% screening; 11.5% diagnostic) were performed in 1,288,886 women; 76,567 (1.9%) were followed by tissue sampling (1.2% screening; 7.1% diagnostic). Tissue sampling frequency following diagnostic mammography increased over time in women over 50 but remained stable following screening mammography. The frequency of invasive cancer increased with age and was more common following a diagnostic (29.3%) vs screening (19.8%) mammogram; the frequency of high risk lesions (ADH; lobular neoplasia) was highest in women aged 50-59. For tissue sampling following screening mammograms, the frequency of DCIS increased over time while benign diagnoses decreased. No significant time trends were noted for diagnoses associated with diagnostic mammograms. Women aged 40-59 with dense breasts and a tissue sampling following screening mammogram had a significantly higher frequency of DCIS (40-49: 4.8% vs 3.2%, P< 0.001; 50-59: 7.0% vs 5.7%, P=0.007). Women aged 40-59 with > 1first degree relative with breast cancer vs none that had a tissue sampling following screening mammogram had a significantly higher frequency of invasive cancer (40-49: 11.4% vs 9.4%, p=0.008; 50-59: 19.8% vs 18.2%, p =0.086) and DCIS (40-49: 6.2% vs 4.0%, p< 0.001; 50-59: 8.2% vs 6.2%, p< 0.001). Conclusions: There was an increase in DCIS and a decrease in benign diagnoses in tissues samples after screening mammography over time. No trends were seen following diagnostic mammography. DCIS was also more frequent in women with dense breasts.

Abstract P4-01-15: Impact of Preoperative MRI on the Surgical Treatment of Breast Cancer: A SEER-Medicare Analysis

Background: Preoperative magnetic resonance imaging (MRI) use has increased for women with invasive breast cancer and ductal carcinoma in situ (DCIS). Prior studies have disputed whether preoperative MRI is associated with increased rates of mastectomy. We evaluated the rates of mastectomy versus breast conserving surgery (BCS) and their association with preoperative MRI in older women.

Methods: We identified women in SEER-Medicare 66 years or older diagnosed with DCIS or invasive breast cancer between 2002 and 2007 treated within 6 months of diagnosis with mastectomy or breast conserving surgery (BCS) with or without radiotherapy (RT). Preoperative MRI was defined as MRI occurring before a woman's first surgery after her initial diagnosis. We looked for surgical treatment (BCS or mastectomy) and radiotherapy, age at diagnosis, year of diagnosis and cancer type, overall and separately by receipt of MRI. We examined the association of surgical treatment with MRI using multivariable logistic regression adjusting for age at diagnosis, year of diagnosis, cancer type and radiotherapy.

Results: Among the 70,758 women identified, 5,126 (7.2%) had a preoperative MRI. The overall use of MRI increased from 1.2% in 2002 to 18.0% in 2007 (p &lt; 0.0001). Women with MRI were more likely to undergo mastectomy than those without MRI (331 per 1000 vs. 314 per 1000; respectively, p &lt; 0.0001) and more likely to undergo BCS RT (432 per 1000 vs. 354 per 1000; respectively, p &lt; 0.0001).

Conclusion: The use of preoperative MRI has increased in recent years and is significantly associated with increased rates of BCS RT and mastectomy. Limitations of this study include that it is an observational analysis and that selection bias may exist for MRI and aggressive treatment that is not related to what is observed on MRI. Further studies are needed to understand how patient characteristics and information obtained from MRI influence treatment choices.

Automated inter-rater reliability assessment and electronic data collection in a multi-center breast cancer study

Background

The choice between paper data collection methods and electronic data collection (EDC) methods has become a key question for clinical researchers. There remains a need to examine potential benefits, efficiencies, and innovations associated with an EDC system in a multi-center medical record review study.

Methods

A computer-based automated menu-driven system with 658 data fields was developed for a cohort study of women aged 65 years or older, diagnosed with invasive histologically confirmed primary breast cancer (N = 1859), at 6 Cancer Research Network sites. Medical record review with direct data entry into the EDC system was implemented. An inter-rater and intra-rater reliability (IRR) system was developed using a modified version of the EDC.

Results

Automation of EDC accelerated the flow of study information and resulted in an efficient data collection process. Data collection time was reduced by approximately four months compared to the project schedule and funded time available for manuscript preparation increased by 12 months. In addition, an innovative modified version of the EDC permitted an automated evaluation of inter-rater and intra-rater reliability across six data collection sites.

Conclusion

Automated EDC is a powerful tool for research efficiency and innovation, especially when multiple data collection sites are involved.

Women's responses to a mailed hormone replacement therapy workbook

OBJECTIVE

Effectively communicating information about the complex decisions that face women at midlife, including whether to use hormone replacement therapy (HRT), is an ongoing challenge. Although numerous decision-making tools exist, few have been evaluated. The objective of this study was to examine women's use of a workbook designed to promote informed HRT decision-making.

DESIGN

We developed a workbook to prepare women to discuss HRT, osteoporosis, heart disease, and breast cancer with their providers. To evaluate the workbook, women aged 45-65 years were randomly assigned to one of three groups: (1) workbook plus baseline and 6-month surveys, (2) workbook and 6-month survey, or (3) no workbook with both surveys. Results are based on the responses of 580 women in groups 1 and 2 (response rate, 84.2%).

RESULTS

At 6 months, 79% of women recalled receiving the workbook, of whom 51% read all or most of it, 35% skimmed or read part of it, and 14% did not read it. The percentages of women completing self-assessments were 55% osteoporosis; 56% heart disease; 58% breast cancer; 57% advantages and disadvantages of HRT; and 52% personal preferences about HRT. As a result of the workbook, 10% made an appointment with their providers, and 12% had a discussion about HRT with their providers. Use of the workbook was not associated with menopause symptoms, attitudes about or use of HRT, hysterectomy, or provider discussions about menopause and HRT.

CONCLUSION

This simple approach of using a mailed workbook holds promise as a successful mechanism to prepare women to discuss HRT and other related health issues with their providers.

What factors account for hormone replacement therapy prescribing frequency?

OBJECTIVES

The purpose of this study was to compare hormone replacement therapy (HRT) prescribing frequency to provider characteristics, attitudes and beliefs about menopause and HRT.

METHODS

There was a mailed survey of providers at a large staff-model HMO in Washington state. Participants included 250 family practice physicians, 22 gynecologists, and 13 women's health care specialists and nurse midwives (83% response rate). The primary outcome, "HRT prescribing frequency" (derived from automated pharmacy and visit data) was defined as: the total number of estrogen prescriptions written by the provider and filled by women aged 50-80 years during the 12 months prior to the survey, divided by the number of visits made to the provider by women aged 50-80 years during that same 12-month period. Covariates included provider characteristics and beliefs about menopause and HRT. Logistic regression was used to distinguish providers in the upper 40% versus the lower 60% of HRT prescribing frequency.

RESULTS

Controlling for age and practice type, HRT prescribing frequency was lower among men than women providers (odds ratio [OR] 0.38, 95% confidence interval [CI] 0.21-0.65), higher among providers who agreed (vs. disagreed or neutral) that a convincing scientific case has been made that HRT prevents heart disease (OR 2.66, 95% CI 1.53-4.61), and higher among those in the upper tertile vs. lower tertiles of an HRT encouragement scale (OR 2.50, 95% CI 1.29-4.85).

CONCLUSIONS

Female providers and providers with positive attitudes toward HRT are the most likely to prescribe it.

Bone mineral density and endogenous hormones and risk of breast cancer in postmenopausal women (United States)

OBJECTIVE

This case-cohort study was designed to examine whether total hip bone mineral density (BMD) is independently associated with breast cancer over and above its association with other determinants, including levels of total and bioavailable estradiol and testosterone and sex-hormone binding globulin.

METHODS

Our study population was selected from a cohort of 8,203 postmenopausal women who were screened for the Fracture Intervention Trial in 1992, at which time BMD was assessed, and blood samples were obtained. A total of 109 women developed breast cancer during four years of follow-up; 173 other randomly selected women from the larger cohort were also selected. Cox proportional hazards with robust variance adjustment was used for these analyses.

RESULTS

Relative to women in the lower fourth of the BMD distribution, the risk associated with being in the upper fourth was 2.6 (95% confidence interval (CI) 1.1-5.8). After adjusting for serum hormone levels, the corresponding relative risk was 2.5 (95% CI 0.9-5.2). With body mass index and number of years since menopause added to the multivariate analysis, the relative risk decreased to 1.4 (95% CI 0.5-4.0).

CONCLUSIONS

BMD may not influence breast cancer risk independent of its relationship with endogenous hormones and measured covariates.

Bone mineral density and breast cancer risk in postmenopausal women

Two recent studies have shown a woman's bone mineral density (BMD) (a composite measure of exposure to many different factors throughout one's lifetime) predicts breast cancer. In a prospective cohort study, we examined whether hip BMD was associated with breast cancer risk among 8203 postmenopausal women. During an average follow-up of 3.7 years, 131 incident breast cancer cases (102 invasive) were identified. Cox proportional hazards models were used to obtain estimates of the relative risk of breast cancer. Our results demonstrate an increase in breast cancer risk among women with higher BMD. Independent of age, geographic area, and body mass index, relative to the lowest BMD quartile the risk of breast cancer (95% confidence interval) by increasing quartile was 1.9 (1.1, 3.2), 1.5 (0.8, 2.6), and 1.5 (0.8, 2.7), respectively. An examination of other factors important in determining BMD may help explain the positive association between BMD and breast cancer.

Cost comparison of mastectomy versus breast-conserving therapy for early-stage breast cancer

BACKGROUND

Choice of treatment for early-stage breast cancer depends on many factors, including the size and stage of the cancer, the woman's age, comorbid conditions, and perhaps the costs of treatment. We compared the costs of all medical care for women with early-stage breast cancer cases treated by breast-conserving therapy (BCT) or mastectomy.

METHODS

A total of 1675 women 35 years old or older with incident early-stage breast cancer were identified in a large regional nonprofit health maintenance organization in the period 1990 through 1997. The women were treated with mastectomy only (n = 183), mastectomy with adjuvant hormonal therapy or chemotherapy (n = 417), BCT with radiation therapy (n = 405), or BCT with radiation therapy and adjuvant hormonal therapy or chemotherapy (n = 670). The costs of all medical care for the period 1990 through 1998 were computed for each woman, and monthly costs were analyzed by treatment, adjusting for age and cancer stage. All statistical tests were two-sided.

RESULTS

At 6 months after diagnosis, the mean total medical care costs for the four groups differed statistically significantly (P:<.001), with BCT being more expensive than mastectomy. The adjusted mean costs were $12 987, $14 309, $14 963, and $15 779 for mastectomy alone, mastectomy with adjuvant therapy, BCT plus radiation therapy, and BCT plus radiation therapy with adjuvant therapy, respectively. At 1 year, the difference in costs was still statistically significant (P:<.001), but costs were influenced more by the use of adjuvant therapy than by type of surgery. The 1-year adjusted mean costs were $16 704, $18 856, $17 344, and $19 081, respectively, for the four groups. By 5 years, BCT was less expensive than mastectomy (P:<.001), with 5-year adjusted mean costs of $41 930, $45 670, $35 787, and $39 926, respectively. Costs also varied by age, with women under 65 years having higher treatment costs than older women.

CONCLUSIONS

BCT may have higher short-term costs but lower long-term costs than mastectomy.

Understanding attitudes of older women toward hormone replacement therapy

BACKGROUND

Counseling women facing the decision to initiate, continue, or discontinue hormone replacement therapy represents a major challenge for providers. Women's attitudes deserve careful consideration in this context, because attitudes may influence hormone replacement therapy use and patients' satisfaction with decision-making. Little is known about factors that may explain different attitudes.

METHODS

To evaluate the association between char acteristics of peri- and postmenopausal women and their attitudes toward hormone replacement therapy, we conducted a population-based, computer-assisted telephone survey of 1,076 randomly selected women, ages 50-80, at a staff-model health maintenance organization. Women with a positive or neutral attitude were compared to those with a negative attitude. We exam ined associations between attitudes and demographic and clinical characteristics, self-rated health status, physical function, personal and family history of condi tions affected by hormone replacement therapy, gyne cologic visits, provider characteristics, interactions with provider, and sources of information about hor mone replacement therapy.

RESULTS

The perception of being adequately informed about the benefits of hormone replacement therapy by one's provider was associated with a tripling of the likelihood of having a positive attitude toward hormone replacement therapy. Additional fac tors associated with positive attitudes included past hormone replacement therapy use, younger age, a higher level of physical functioning, and personal history of heart disease. Relationships between these vari ables and attitudes varied among current hormone replacement therapy users and nonusers.

CONCLUSIONS

The study findings reinforce the critical role of provider counseling in shaping women's atti tudes about hormone replacement therapy.

Inclusion of older women in randomized clinical trials: factors associated with taking study medication in the fracture intervention trial

OBJECTIVES

The purpose of this study is to describe the early study medication discontinuation (SMD) experience during the first year of follow-up in a randomized clinical trial in older women and to determine the associations between various baseline characteristics and risk of SMD. DESIGH, SETTING, AND PARTICIPANTS: The authors studied 6,459 postmenopausal women aged 55 to 80 from 11 clinical settings during their first year of participation in the Fracture Intervention Trial (FIT). This trial was designed to test the efficacy of alendronate (Fosamax) on fracture prevention among women with low bone mass.

RESULTS

Study medication discontinuation was greatest in the first month post-randomization (2.2%) and declined thereafter. Cumulative rates of study medication discontinuation were 4.8% (n = 311) at 3 months and 11.1% (n = 717) at 12 months. SMD was not associated with age, marital status, alcohol consumption, regular exercise, past estrogen replacement therapy use, bone mineral density, or personal or maternal fracture history. After adjusting for covariates and markers of comorbidity, the strongest predictor of SMD was fair-to-poor self-rated health (relative risk (RR) 2.10; 95% confidence interval (CI) 1.47, 2.99). Having four or more depressive symptoms was also a significant predictor and had a risk associated with SMD (RR vs none 1.34; 95% CI 1.05, 1.71) similar to that seen for individuals with good self-rated health (RR 1.49; 95% CI 1.16, 1.91).

CONCLUSIONS

Results from this cohort emphasize that clinical trials in older women with multiple concomitant conditions can achieve high levels of adherence. Thought should be given to measuring self-rated health and depressive symptoms before randomization to help identify individuals to be targeted for special assistance programs that focus on encouraging adherence.

Inhibitors of hydroxymethylglutaryl-coenzyme A reductase and risk of fracture among older women

BACKGROUND

Inhibitors of hydroxymethylglutaryl-coenzyme A reductase (statins) increase new bone formation in rodents and in human cells in vitro. Statin use is associated with increased bone mineral density of the femoral neck. We undertook a population-based case-control study at six health-maintenance organisations in the USA to investigate further the relation between statin use and fracture risk among older women.

METHODS

We investigated women aged 60 years or older. Exposure, outcome, and confounder information was obtained from automated claims and pharmacy data from October, 1994, to September, 1997. Cases had an incident diagnosis of non-pathological fracture of the hip, humerus, distal tibia, wrist, or vertebrae between October, 1996, and September, 1997. Controls had no fracture during this period. We excluded women with records of dispensing of drugs to treat osteoporosis.

FINDINGS

There were 928 cases and 2747 controls. Compared with women who had no record of statin dispensing during the previous 2 years, women with 13 or more statin dispensings during this period had a decreased risk of non-pathological fracture (odds ratio 0.48 [95% CI 0.27-0.83]) after adjustment for age, number of hospital admissions during the previous year, chronic disease score, and use of non-statin lipid-lowering drugs. No association was found between fracture risk and fewer than 13 dispensings of statins or between fracture risk and use of non-statin lipid-lowering drugs.

INTERPRETATION

Statins seem to be protective against non-pathological fracture among older women. These findings are compatible with the hypothesis that statins increase bone mineral density in human beings and thereby decrease the risk of osteoporotic fractures.

Perceived barriers and recommendations concerning hormone replacement therapy counseling among primary care providers

OBJECTIVE

To increase our understanding of the factors that impede or promote counseling about hormone replacement therapy, we asked clinicians to provide information concerning barriers and strategies to promote counseling.

DESIGN

We asked clinicians to consider two different scenarios: (1) what they do in they current practice and (2) what they would do if their health care systems implemented the United States Preventive Services Task Force recommendation regarding hormone replacement therapy counseling. A total of 49 of 50 invited clinicians participated in one of six focus group interviews (three women's groups and three men's groups). Our analysis consisted of four steps: (1) identifying segments and classifying them into themes, (2) categorizing themes into topic areas, (3) establishing a final consensus of themes and topics, and (4) ascertaining similarities and contrasts among groups. Transcripts of sessions were analyzed across groups for themes using a text-based analysis system. Conceptualization of themes was derived using a system model of preventive care. Interrater agreement before consensus was good: Kappa (kappa) ranged from 0.70 to 1.00.

RESULTS

For current practice, identified barriers included lack of information about risks and benefits, unique challenges of counseling, and lack of resources to conduct counseling. The major strategies suggested were to develop and distribute patient education materials. Discussions about barriers to implementing the United States Task Force recommendation focused on lack of information and resources.

CONCLUSIONS

Suggested strategies were multiple, involving individual-, relationship-, and system-level interventions. We expect the strategies identified to be supportive of future efforts to promote counseling for hormone replacement therapy.

Are long-term hormone replacement therapy users different from short-term and never users?

The characteristics that differentiate long-term (> or = 10 years) hormone replacement therapy (HRT) users from short-term (<10 years) users and nonusers are not well documented. The epidemiology of long-term HRT use was investigated within a random sample survey of 703 women aged 50-80 years who were members of Group Health Cooperative (GHC) of Puget Sound. Women who had been menopausal for > or = 10 years comprised the study population. Long-term HRT users (29.4 percent) were compared with short-term (28.1 percent) and never users (42.5 percent). The authors examined the association between duration of HRT use and demographic characteristics, personal and family medical history, menopausal symptoms, information used in decision making, attitudes toward HRT, provider encouragement to use HRT, and GHC utilization. Compared with never users, the strongest correlates of long-term HRT use were having a hysterectomy before or after menopause, positive attitudes espousing the benefits of HRTs, and perceived provider encouragement to use HRT. Long-term HRT use was not associated with educational attainment, ethnicity, body mass index, health status, physical activity, or family medical history. Correlates commonly associated with HRT use, such as higher education, greater physical activity and functioning, and lower chronic disease comorbidity, did not significantly distinguish long-term from short-term users.