Cost-effectiveness of different reading and referral strategies in mammography screening in the Netherlands

Modeling Radiologists' Assessments to Explore Pairing Strategies for Optimized Double Reading of Screening Mammograms

PURPOSE

To develop a model that simulates radiologist assessments and use it to explore whether pairing readers based on their individual performance characteristics could optimize screening performance.

METHODS

Logistic regression models were designed and used to model individual radiologist assessments. For model evaluation, model-predicted individual performance metrics and paired disagreement rates were compared against the observed data using Pearson correlation coefficients. The logistic regression models were subsequently used to simulate different screening programs with reader pairing based on individual true-positive rates (TPR) and/or false-positive rates (FPR). For this, retrospective results from breast cancer screening programs employing double reading in Sweden, England, and Norway were used. Outcomes of random pairing were compared against those composed of readers with similar and opposite TPRs/FPRs, with positive assessments defined by either reader flagging an examination as abnormal.

RESULTS

The analysis data sets consisted of 936,621 (Sweden), 435,281 (England), and 1,820,053 (Norway) examinations. There was good agreement between the model-predicted and observed radiologists' TPR and FPR (r ≥ 0.969). Model-predicted negative-case disagreement rates showed high correlations (r ≥ 0.709), whereas positive-case disagreement rates had lower correlation levels due to sparse data (r ≥ 0.532). Pairing radiologists with similar FPR characteristics (Sweden: 4.50% [95% confidence interval: 4.46%-4.54%], England: 5.51% [5.47%-5.56%], Norway: 8.03% [7.99%-8.07%]) resulted in significantly lower FPR than with random pairing (Sweden: 4.74% [4.70%-4.78%], England: 5.76% [5.71%-5.80%], Norway: 8.30% [8.26%-8.34%]), reducing examinations sent to consensus/arbitration while the TPR did not change significantly. Other pairing strategies resulted in equal or worse performance than random pairing.

CONCLUSIONS

Logistic regression models accurately predicted screening mammography assessments and helped explore different radiologist pairing strategies. Pairing readers with similar modeled FPR characteristics reduced the number of examinations unnecessarily sent to consensus/arbitration without significantly compromising the TPR.

HIGHLIGHTS

A logistic-regression model can be derived that accurately predicts individual and paired reader performance during mammography screening reading.Pairing screening mammography radiologists with similar false-positive characteristics reduced false-positive rates with no significant loss in true positives and may reduce the number of examinations unnecessarily sent to consensus/arbitration.

[Measurements errors in screening mammogram interpretation by radiologists]

The timely detection of breast cancer is achieved through mammography; however, the quality of the procedure should be addressed for proper performance and interpretation. Despite recent improvements in quality assurance in mammography, interpretation still depends on each reader; therefore, errors can be made when interpreting screening mammograms, leading to unnecessary biopsies and/or overdiagnosis, with sustained physical, economic and psychological consequences. Since interpretation is related to the perceptive and cognitive ability of the radiologist, it is necessary to have extensive knowledge about the possible errors that may occur during interpretation, as well as of the way how they can be reduced, prevented and/or corrected to provide the patient with the highest possible level of safety.

Can eye-tracking metrics be used to better pair radiologists in a mammogram reading task?

PURPOSE

To propose a framework for optimal pairing of radiologists when reading mammograms based on their search patterns.

MATERIALS AND METHODS

Four experienced and four less-experienced radiologists were asked to assess 120 cases (59 with cancers) while their eye positions were tracked. Fourteen eye-tracking metrics were extracted to quantify the differences among radiologists' visual search pattern. For each radiologist and metric, less-experienced radiologists and expert readers were ranked based on the level of similarities in gaze patterns (from the most different to the most similar). Less-experienced readers and experts were also ranked based on the values of area under the receiver operating characteristic curve (AUC) after pairing (the best possible way of ranking). Using the Kendall's tau distance, rankings based on different metrics were compared with the best possible ranking. Using paired Wilcoxon signed-rank test, the AUC values when pairing in the best way were compared with pairing based on different metrics. Finally, we investigated the robustness of pairing strategies against the small sample size.

RESULTS

For ranking the experienced radiologists, results from eight metrics were as good as the best possible ranking. For the less-experienced ones, only one metric resulted in a ranking comparable to the best possible way of ranking. The AUC values of pairings based on these metrics did not differ significantly from the best pairing scenario. Compared to the pairings based on the cognitive metrics, the ranking based on AUC values varied more greatly with the sample size, suggesting that it is less robust against the small sample size compared to the cognitive metrics.

CONCLUSION

Different pairings may have different effects on performance; some are detrimental while some improve the performance of the pair. Using the suggested cognitive metrics, we can optimize the pairings even with a small dataset.

Effectiveness and cost-effectiveness of double reading in digital mammography screening: A systematic review and meta-analysis

PURPOSE

Double reading is the strategy of choice for mammogram interpretation in screening programmes. It remains, however, unknown whether double reading is still the strategy of choice in the context of digital mammography. Our aim was to determine the effectiveness and cost-effectiveness of double reading versus single reading of digital mammograms in screening programmes.

METHODS

We performed a systematic review by searching the PubMed, Embase, and Cochrane Library databases up to April 2017. We used the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies) tool and CHEERS (Consolidated Health Economic Evaluation Reporting Standards) checklist to assess the methodological quality of the diagnostic studies and economic evaluations, respectively. A proportion's meta-analysis approach, 95% Confidence Intervals (95% CI) and test of heterogeneity (P values) were used for pooled results. Costs are expressed US$ PPP (United States Dollar purchasing power parities). The PROSPERO ID of this Systematic Review's protocol is CRD42014013804.

RESULTS

Of 1473 potentially relevant hits, four high-quality studies were included. The pooled cancer detection rate of double reading was 6.01 per 1000 screens (CI: 4.47‰-7.77‰), and it was 5.65 per 1000 screens (CI: 3.95‰-7.65‰) for single reading (P=0.76). The pooled proportion of false-positives of double reading was 47.03 per 1000 screens (CI: 39.13‰-55.62‰) and it was 40.60 per 1000 screens (CI: 38.58‰-42.67‰) for single reading (P=0.12). One study reported, for double reading, an ICER (Incremental Cost-Effectiveness Ratio) of 16,684 Euros (24,717 US$ PPP; 2015 value) per detected cancer. Single reading+CAD (computer-aided-detection) was cost-effective in Japan.

CONCLUSION

The evidence of benefit for double reading compared to single reading for digital mammography interpretation is scarce. Double reading seems to increase operational costs, have a not significantly higher false-positive rate, and a similar cancer detection rate.

Improvement in risk prediction, early detection and prevention of breast cancer in the NHS Breast Screening Programme and family history clinics: a dual cohort study

Background

In the UK, women are invited for 3-yearly mammography screening, through the NHS Breast Screening Programme (NHSBSP), from the ages of 47–50 years to the ages of 69–73 years. Women with family histories of breast cancer can, from the age of 40 years, obtain enhanced surveillance and, in exceptionally high-risk cases, magnetic resonance imaging. However, no NHSBSP risk assessment is undertaken. Risk prediction models are able to categorise women by risk using known risk factors, although accurate individual risk prediction remains elusive. The identification of mammographic breast density (MD) and common genetic risk variants [single nucleotide polymorphisms (SNPs)] has presaged the improved precision of risk models.

Objectives

To (1) identify the best performing model to assess breast cancer risk in family history clinic (FHC) and population settings; (2) use information from MD/SNPs to improve risk prediction; (3) assess the acceptability and feasibility of offering risk assessment in the NHSBSP; and (4) identify the incremental costs and benefits of risk stratified screening in a preliminary cost-effectiveness analysis.

Design

Two cohort studies assessing breast cancer incidence.

Setting

High-risk FHC and the NHSBSP Greater Manchester, UK.

Participants

A total of 10,000 women aged 20–79 years [Family History Risk Study (FH-Risk); UK Clinical Research Network identification number (UKCRN-ID) 8611] and 53,000 women from the NHSBSP [aged 46–73 years; Predicting the Risk of Cancer At Screening (PROCAS) study; UKCRN-ID 8080].

Interventions

Questionnaires collected standard risk information, and mammograms were assessed for breast density by a number of techniques. All FH-Risk and 10,000 PROCAS participants participated in deoxyribonucleic acid (DNA) studies. The risk prediction models Manual method, Tyrer–Cuzick (TC), BOADICEA (Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm) and Gail were used to assess risk, with modelling based on MD and SNPs. A preliminary model-based cost-effectiveness analysis of risk stratified screening was conducted.

Main outcome measures

Breast cancer incidence.

Data sources

The NHSBSP; cancer registration.

Results

A total of 446 women developed incident breast cancers in FH-Risk in 97,958 years of follow-up. All risk models accurately stratified women into risk categories. TC had better risk precision than Gail, and BOADICEA accurately predicted risk in the 6268 single probands. The Manual model was also accurate in the whole cohort. In PROCAS, TC had better risk precision than Gail [area under the curve (AUC) 0.58 vs. 0.54], identifying 547 prospective breast cancers. The addition of SNPs in the FH-Risk case–control study improved risk precision but was not useful inBRCA1(breast cancer 1 gene) families. Risk modelling of SNPs in PROCAS showed an incremental improvement from using SNP18 used in PROCAS to SNP67. MD measured by visual assessment score provided better risk stratification than automatic measures, despite wide intra- and inter-reader variability. Using a MD-adjusted TC model in PROCAS improved risk stratification (AUC = 0.6) and identified significantly higher rates (4.7 per 10,000 vs. 1.3 per 10,000;p < 0.001) of high-stage cancers in women with above-average breast cancer risks. It is not possible to provide estimates of the incremental costs and benefits of risk stratified screening because of lack of data inputs for key parameters in the model-based cost-effectiveness analysis.

Conclusions

Risk precision can be improved by using DNA and MD, and can potentially be used to stratify NHSBSP screening. It may also identify those at greater risk of high-stage cancers for enhanced screening. The cost-effectiveness of risk stratified screening is currently associated with extensive uncertainty. Additional research is needed to identify data needed for key inputs into model-based cost-effectiveness analyses to identify the impact on health-care resource use and patient benefits.

Future work

A pilot of real-time NHSBSP risk prediction to identify women for chemoprevention and enhanced screening is required.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme. The DNA saliva collection for SNP analysis for PROCAS was funded by the Genesis Breast Cancer Prevention Appeal.

Cost-Effectiveness of Double Reading versus Single Reading of Mammograms in a Breast Cancer Screening Programme

OBJECTIVES

The usual practice in breast cancer screening programmes for mammogram interpretation is to perform double reading. However, little is known about its cost-effectiveness in the context of digital mammography. Our purpose was to evaluate the cost-effectiveness of double reading versus single reading of digital mammograms in a population-based breast cancer screening programme.

METHODS

Data from 28,636 screened women was used to establish a decision-tree model and to compare three strategies: 1) double reading; 2) double reading for women in their first participation and single reading for women in their subsequent participations; and 3) single reading. We calculated the incremental cost-effectiveness ratio (ICER), which was defined as the expected cost per one additionally detected cancer. We performed a deterministic sensitivity analysis to test the robustness of the ICER.

RESULTS

The detection rate of double reading (5.17‰) was similar to that of single reading (4.78‰; P = .768). The mean cost of each detected cancer was €8,912 for double reading and €8,287 for single reading. The ICER of double reading versus single reading was €16,684. The sensitivity analysis showed variations in the ICER according to the sensitivity of reading strategies. The strategy that combines double reading in first participation with single reading in subsequent participations was ruled out due to extended dominance.

CONCLUSIONS

From our results, double reading appears not to be a cost-effective strategy in the context of digital mammography. Double reading would eventually be challenged in screening programmes, as single reading might entail important net savings without significantly changing the cancer detection rate. These results are not conclusive and should be confirmed in prospective studies that investigate long-term outcomes like quality adjusted life years (QALYs).

Towards personalized screening: Cumulative risk of breast cancer screening outcomes in women with and without a first-degree relative with a history of breast cancer

Several reviews have estimated the balance of benefits and harms of mammographic screening in the general population. The balance may, however, differ between individuals with and without family history. Therefore, our aim is to assess the cumulative risk of screening outcomes; screen-detected breast cancer, interval cancer, and false-positive results, in women screenees aged 50-75 and 40-75, with and without a first-degree relative with a history of breast cancer at the start of screening. Data on screening attendance, recall and breast cancer detection were collected for each woman living in Nijmegen (The Netherlands) since 1975. We used a discrete time survival model to calculate the cumulative probability of each major screening outcome over 19 screening rounds. Women with a family history of breast cancer had a higher risk of all screening outcomes. For women screened from age 50-75, the cumulative risk of screen-detected breast cancer, interval cancer and false-positive results were 9.0, 4.4 and 11.1% for women with a family history and 6.3, 2.7 and 7.3% for women without a family history, respectively. The results for women 40-75 followed the same pattern for women screened 50-75 for cancer outcomes, but were almost doubled for false-positive results. To conclude, women with a first-degree relative with a history of breast cancer are more likely to experience benefits and harms of screening than women without a family history. To complete the balance and provide risk-based screening recommendations, the breast cancer mortality reduction and overdiagnosis should be estimated for family history subgroups.

Recommendations for breast cancer surveillance for female survivors of childhood, adolescent, and young adult cancer given chest radiation: a report from the International Late Effects of Childhood Cancer Guideline Harmonization Group

Female survivors of childhood, adolescent, and young adult (CAYA) cancer who were given radiation to fields that include breast tissue (ie, chest radiation) have an increased risk of breast cancer. Clinical practice guidelines are essential to ensure that these individuals receive optimum care and to reduce the detrimental consequences of cancer treatment; however, surveillance recommendations vary among the existing long-term follow-up guidelines. We applied evidence-based methods to develop international, harmonised recommendations for breast cancer surveillance among female survivors of CAYA cancer who were given chest radiation before age 30 years. The recommendations were formulated by an international, multidisciplinary panel and are graded according to the strength of the underlying evidence.

Trends in incidence and detection of advanced breast cancer at biennial screening mammography in The Netherlands: a population based study

Introduction

The aims of this study were to determine trends in the incidence of advanced breast cancer at screening mammography and the potential of screening to reduce it.

Methods

We included a consecutive series of 351,009 screening mammograms of 85,274 women aged 50-75 years, who underwent biennial screening at a Dutch breast screening region in the period 1997-2008. Two screening radiologists reviewed the screening mammograms of all advanced screen detected and advanced interval cancers and determined whether the advanced cancer (tumor > 20 mm and/or lymph node positive tumor) had been visible at a previous screen. Interval cancers were breast cancers diagnosed in women after a negative screening examination (defined as no recommendation for referral) and before any subsequent screen. Patient and tumor characteristics were compared between women with advanced cancer and women with non-advanced cancer, including ductal carcinoma in situ.

Results

A total of 1,771 screen detected cancers and 669 interval cancers were diagnosed in 2,440 women. Rates of advanced cancer remained stable over the 12-year period; the incidence of advanced screen-detected cancers fluctuated between 1.5 - 1.9 per 1,000 screened women (mean 1.6 per 1,000) and of advanced interval cancers between 0.8 - 1.6 per 1,000 screened women (mean 1.2 per 1,000). Of the 570 advanced screen-detected cancers, 106 (18.6%) were detected at initial screening; 265 (46.5%) cancers detected at subsequent screening had been radiologically occult at the previous screening mammogram, 88 (15.4%) had shown a minimal sign, and 111 (19.5%) had been missed. Corresponding figures for advanced interval cancers were 50.9% (216/424), 24.3% (103/424) and 25.1% (105/424), respectively. At multivariate analysis, women with a ≥ 30 months interval between the latest two screens had an increased risk of screen-detected advanced breast cancer (OR 1.63, 95%CI: 1.07-2.48) and hormone replacement therapy increased the risk of advanced disease among interval cancers (OR 3.04, 95%CI: 1.22-7.53).

Conclusion

We observed no decline in the risk of advanced breast cancer during 12 years of biennial screening mammography. The majority of these cancers could not have been prevented through earlier detection at screening.

Feasibility of noncontact piezoelectric detection of photoacoustic signals in tissue-mimicking phantoms

The feasibility of air-coupled ultrasound transducers to detect laser-induced ultrasound from artificial blood vessels embedded in an optically scattering phantom is demonstrated. These air-coupled transducers allow new applications in biomedical photoacoustic imaging where contact with tissue is not preferred. One promising application of such transducers is the addition of photoacoustic imaging to the regular x-ray mammographic screening procedure.

Inter-observer variability in mammography screening and effect of type and number of readers on screening outcome

We prospectively determined the variability in radiologists' interpretation of screening mammograms and assessed the influence of type and number of readers on screening outcome. Twenty-one screening mammography radiographers and eight screening radiologists participated. A total of 106 093 screening mammograms were double-read by two radiographers and, in turn, by two radiologists. Initially, radiologists were blinded to the referral opinion of the radiographers. A woman was referred if she was considered positive at radiologist double-reading with consensus interpretation or referred after radiologist review of positive cases at radiographer double-reading. During 2-year follow-up, clinical data, breast imaging reports, biopsy results and breast surgery reports were collected of all women with a positive screening result from any reader. Single radiologist reading (I) resulted in a mean cancer detection rate of 4.64 per 1000 screens (95% confidence intervals (CI)=4.23–5.05) with individual variations from 3.44 (95% CI=2.30–4.58) to 5.04 (95% CI=3.81–6.27), and a sensitivity of 63.9% (95% CI=60.5–67.3), ranging from 51.5% (95% CI=39.6–63.3) to 75.0% (95% CI=65.3–84.7). Sensitivity at non-blinded, radiologist double-reading (II), radiologist double-reading followed by radiologist review of positive cases at radiographer double-reading (III), triple reading by one radiologist and two radiographers with referral of all positive readings (IV) and quadruple reading by two radiologists and two radiographers with referral of all positive readings (V) were as follows: 68.6% (95% CI=65.3–71.9) (II); 73.2% (95% CI=70.1–76.4) (III); 75.2% (95% CI=72.1–78.2) (IV), and 76.9% (95% CI=73.9–79.9) (V). We conclude that screener performance significantly varied at single-reading. Double-reading increased sensitivity by a relative 7.3%. When there is a shortage of screening radiologists, triple reading by one radiologist and two radiographers may replace radiologist double-reading.

Cost-effectiveness of opportunistic versus organised mammography screening in Switzerland

BACKGROUND

Various centralised mammography screening programmes have shown to reduce breast cancer mortality at reasonable costs. However, mammography screening is not necessarily cost-effective in every situation. Opportunistic screening, the predominant screening modality in several European countries, may under certain circumstances be a cost-effective alternative. In this study, we compared the cost-effectiveness of both screening modalities in Switzerland.

METHODS

Using micro-simulation modelling, we predicted the effects and costs of biennial mammography screening for 50-69 years old women between 1999 and 2020, in the Swiss female population aged 30-70 in 1999. A sensitivity analysis on the test sensitivity of opportunistic screening was performed.

RESULTS

Organised mammography screening with an 80% participation rate yielded a breast cancer mortality reduction of 13%. Twenty years after the start of screening, the predicted annual breast cancer mortality was 25% lower than in a situation without screening. The 3% discounted cost-effectiveness ratio of organised mammography screening was euro11,512 per life year gained. Opportunistic screening with a similar participation rate was comparably effective, but at twice the costs: euro22,671-24,707 per life year gained. This was mainly related to the high costs of opportunistic mammography and frequent use of imaging diagnostics in combination with an opportunistic mammogram.

CONCLUSION

Although data on the performance of opportunistic screening are limited, both opportunistic and organised mammography screening seem effective in reducing breast cancer mortality in Switzerland. However, for opportunistic screening to become equally cost-effective as organised screening, costs and use of additional diagnostics should be reduced.

Additional Double Reading of Screening Mammograms by Radiologic Technologists: Impact on Screening Performance Parameters

BACKGROUND

Studies have shown that having mammography technologists review screening mammograms in addition to radiologist review may increase the number of breast cancers that are detected at screening mammography. We prospectively examined the effects on screening performance of adding independent double reading of screening mammograms by technologists to standard double reading by radiologists.

METHODS

Twenty-one screening mammography technologists and eight certified screening radiologists participated in this study. From January 1, 2003, to January 1, 2005, all 61,251 screening mammograms obtained at two mammography screening units in The Netherlands were independently read (although the second reader was not blinded to the first reader's interpretation) by two technologists and, in turn, by two radiologists. Radiologists were blinded to the referral opinion of the technologists. During a 2-year follow-up period, we collected clinical data, breast imaging reports, biopsy results, and breast surgery reports of all women with a positive screening result (i.e., those that required additional imaging) from any reader. The distributions of categorical variables between subgroups were compared using chi-square or Fisher's exact tests. Differences in referral and detection by radiologists and technologists were assessed using McNemar's test. All statistical tests were two-sided.

RESULTS

The radiologists referred 905 women (referral rate = 1.48%, 95% confidence interval [CI] = 1.38% to 1.57%), of whom 323 had breast cancer, corresponding to a positive predictive value of referral (PPV) of 35.7% (95% CI = 32.6% to 38.8%). Review of 446 additional technologist-positive readings led to another 80 referrals, resulting in the detection of 22 additional cancers. These extra referrals increased the initial referral rate from 1.48% to 1.61% (difference = 0.13%; 95% CI = 0.10% to 0.16%) and the cancer detection rate (CDR) from 5.27 to 5.63 cancers per 1000 women screened (difference = 0.36 cancers per 1000 women screened; 95% CI = 0.24 to 0.55). With technologist double reading only, 829 women would have been referred (referral rate = 1.35%, 95% CI = 1.26% to 1.45%); among these women, 286 cancers were diagnosed (PPV = 34.5%, 95% CI = 31.3% to 37.7%; CDR = 4.67 cancers per 1000 women screened, 95% CI = 4.13 to 5.21). Referral of all 1351 radiologist- and/or technologist-positive readings would have led to 362 cancers found at screening. The cancer detection rate for radiologist double reading would have increased from 5.27 to 5.91 cancers per 1000 women screened (relative increase = 12.1%, 95% CI = 8.8% to 16.5%; difference = 0.64 cancers per 1000 women screened, 95% CI = 0.47 to 0.87).

CONCLUSION

A referral strategy that includes all technologist-positive readings, which would have increased the cancer detection rate while maintaining a low referral rate, should be considered.