Systematically higher Ki67 scores on core biopsy samples compared to corresponding resection specimen in breast cancer: a multi-operator and multi-institutional study

Ki67 has potential clinical importance in breast cancer but has yet to see broad acceptance due to inter-laboratory variability. Here we tested an open source and calibrated automated digital image analysis (DIA) platform to: (i) investigate the comparability of Ki67 measurement across corresponding core biopsy and resection specimen cases, and (ii) assess section to section differences in Ki67 scoring. Two sets of 60 previously stained slides containing 30 core-cut biopsy and 30 corresponding resection specimens from 30 estrogen receptor-positive breast cancer patients were sent to 17 participating labs for automated assessment of average Ki67 expression. The blocks were centrally cut and immunohistochemically (IHC) stained for Ki67 (MIB-1 antibody). The QuPath platform was used to evaluate tumoral Ki67 expression. Calibration of the DIA method was performed as in published studies. A guideline for building an automated Ki67 scoring algorithm was sent to participating labs. Very high correlation and no systematic error (p = 0.08) was found between consecutive Ki67 IHC sections. Ki67 scores were higher for core biopsy slides compared to paired whole sections from resections (p ≤ 0.001; median difference: 5.31%). The systematic discrepancy between core biopsy and corresponding whole sections was likely due to pre-analytical factors (tissue handling, fixation). Therefore, Ki67 IHC should be tested on core biopsy samples to best reflect the biological status of the tumor.

Abstract P5-02-01: Analytical validation and prognostic potential of an automated digital scoring protocol for Ki67: An International Ki67 Working Group study

Background: The nuclear proliferation biomarker Ki67 has multiple potential roles in breast cancer, including aiding decisions based on prognosis, but has unacceptable between-laboratory variability. Here we tested an open source and calibrated automated digital image analysis (DIA) platform to: (i) Assess inter-laboratory reproducibility of automated Ki67 measurement among 17 participating labs and compare those with standardized pathologist-based visual scoring. (ii) Investigate the comparability of Ki67 measurement across corresponding core biopsy and whole section cases. (iii) Test prognostic potential of the built Ki67 scoring algorithms on an independent cohort.

Methods: Two sets of 60 previously stained slides containing 30 core-cut biopsy and 30 corresponding whole tumor sections from 30 ER+ breast cancer cases were sent to 17 participating labs for automated assessment of average Ki67 expression. The blocks were centrally cut and stained for Ki67 using the Mib-1 antibody. The QuPath (open-source software) DIA platform was used to evaluate tumoral Ki67 expression. Calibration of the DIA method was performed in our previous study (Acs et al, Lab Invest 2019). A detailed guideline for building an automated Ki67 scoring algorithm was sent to the participating labs. Visual scoring of average Ki67 expression was performed by pathologists according to published standardized methods (Leung et al, NPJ Br Cancer 2016; Leung et al, Histopath 2019). Locked down DIA Ki67 scoring algorithms were applied to a validation cohort: 222 breast cancer cases from the Karolinska University Hospital in whole section format. Sufficient reproducibility to declare analytical validity was defined as an Intra Class Correlation (ICC) with lower limit of 95% credible interval (CI) >0.80. Markov Chain Monte Carlo routines for generalized linear mixed models were used to estimate ICCs and calculate corresponding CIs.

Results: The same-section ICC was 0.902 (CI: 0.852-0.949) across 17 labs using calibrated DIA platform on core biopsy slides and 0.845 (CI: 0.778-0.912) on whole sections. The different-section ICC across the 17 labs was 0.873 (CI: 0.806-0.932) scoring on core biopsy slides and 0.777 (CI: 0.670-0.874) on whole sections. The pathologist-based visual Ki67 scoring showed ICC of 0.860 for all comparisons, respectively (CI: 0.795-0.927). Similar to what was observed for visual Ki67 scoring, the DIA scores are higher for core biopsy slides compared to paired whole sections (p≤0.001; median difference: 5.31%; IQR: 11.50%). Ki67 scores of all locked down DIA algorithms correlates significantly (p≤0.023) with outcome on the validation cohort (observed hazard ratios range: 2.518-2.922).

Conclusions: Automated Ki67 evaluation using a calibrated, open-source DIA platform (QuPath) met the pre-specified criterion of success on core biopsies but not on whole sections in the multi-institutional setting. The systematic discrepancy between core biopsy and corresponding whole sections was likely due to pre-analytical factors (tissue handling, fixation) and intratumor heterogeneity. We found that different algorithms built according to calibrated DIA methods had similar prognostic potential. Assessment of clinical utility is planned.

Citation Format: Balazs Acs, Samuel C.Y. Leung, Kelley M. Kidwell, Indu Arun, Renaldas Augulis, Sunil S. Badve, Yalai Bai, Anita L. Bane, John M.S. Bartlett, Jane Bayani, Gilbert Bigras, Annika Blank, Signe Borgquist, Henk Buikema, Martin C. Chang, Robin L. Dietz, Andrew Dodson, Anna Ehinger, Susan Fineberg, Cornelia M. Focke, Dongxia Gao, Allen M. Gown, Carolina Gutierrez, Johan Hartman, Judith C. Hugh, Zuzana Kos, Anne-Vibeke Lænkholm, Arvydas Laurinavicius, Richard M. Levenson, Rustin Mahboubi-Ardakani, Mauro G. Mastropasqua, Takuya Moriya, Sharon Nofech-Mozes, C. Kent Osborne, Liron Pantanowitz, Frédérique M. Penault-Llorca, Tammy Piper, Mary Anne Quintayo, Tilman T. Rau, Stefan Reinhard, Stephanie Robertson, Takashi Sakatani, Roberto Salgado, Melanie Spears, Jane Starczynski, Tomoharu Sugie, Bert van der Vegt, Giuseppe Viale, Shakeel Virk, Lila A. Zabaglo, Daniel F. Hayes, Mitch Dowsett, Torsten O. Nielsen, David L. Rimm, International Ki67 in Breast Cancer Working Group, BIG-NABCG. Analytical validation and prognostic potential of an automated digital scoring protocol for Ki67: An International Ki67 Working Group study [abstract]. In: Proceedings of the 2019 San Antonio Breast Cancer Symposium; 2019 Dec 10-14; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2020;80(4 Suppl):Abstract nr P5-02-01.

Analytical validation of a standardised scoring protocol for Ki67 immunohistochemistry on breast cancer excision whole sections: an international multicentre collaboration

AIMS

The nuclear proliferation marker Ki67 assayed by immunohistochemistry has multiple potential uses in breast cancer, but an unacceptable level of interlaboratory variability has hampered its clinical utility. The International Ki67 in Breast Cancer Working Group has undertaken a systematic programme to determine whether Ki67 measurement can be analytically validated and standardised among laboratories. This study addresses whether acceptable scoring reproducibility can be achieved on excision whole sections.

METHODS AND RESULTS

Adjacent sections from 30 primary ER⁺ breast cancers were centrally stained for Ki67 and sections were circulated among 23 pathologists in 12 countries. All pathologists scored Ki67 by two methods: (i) global: four fields of 100 tumour cells each were selected to reflect observed heterogeneity in nuclear staining; (ii) hot-spot: the field with highest apparent Ki67 index was selected and up to 500 cells scored. The intraclass correlation coefficient (ICC) for the global method [confidence interval (CI) = 0.87; 95% CI = 0.799-0.93] marginally met the prespecified success criterion (lower 95% CI ≥ 0.8), while the ICC for the hot-spot method (0.83; 95% CI = 0.74-0.90) did not. Visually, interobserver concordance in location of selected hot-spots varies between cases. The median times for scoring were 9 and 6 min for global and hot-spot methods, respectively.

CONCLUSIONS

The global scoring method demonstrates adequate reproducibility to warrant next steps towards evaluation for technical and clinical validity in appropriate cohorts of cases. The time taken for scoring by either method is practical using counting software we are making publicly available. Establishment of external quality assessment schemes is likely to improve the reproducibility between laboratories further.

Abstract P2-03-01: Analytical validation of a standardized scoring protocol for Ki67 assessed on breast excision whole sections: An international multicenter collaboration

Aims: (i) Determine whether between-observer reproducibility for Ki67 when assessed on whole sections according to a standardized scoring protocol is adequate for clinical application. (ii) Compare between-observer reproducibility of Ki67 scores assessed on hot-spots to scores using a global method that averages across a tissue section.

Background: The nuclear proliferation biomarker Ki67 has multiple potential roles in breast cancer, including aiding decisions based on prognosis, but unacceptable levels of between-laboratory variability have been observed. The International Ki67 in Breast Cancer Working Group has undertaken a systematic program to determine whether Ki67 measurement can be analytically validated and standardized across labs. In phase 1, variability in visual interpretation was identified as an important source of variability. Phases 2 and 3a showed that adherence to defined scoring methods substantially improved reproducibility in scoring tissue microarrays and core-cut biopsies. We now assess whether acceptable reproducibility can be achieved on whole sections.

Methods: Adjacent sections from 30 primary ER+ breast cancers were centrally stained for Ki67 to assemble 4 sets of 30 stained tumor sections, circulated around 23 labs in 12 countries. Ki67 was scored by 2 methods by all labs: (a) global: 4 fields of 100 tumor cells each were selected to reflect observed heterogeneity in nuclear staining (b) hot-spot: the field with highest Ki67 percentage of tumor cells with nuclear staining was selected and up to 500 cells scored. Ki67 scores were log2-transformed for statistical analyses and back-transformed for presentation. The primary objective was to assess whether either method could achieve an intraclass correlation coefficient (ICC) significantly greater than 0.8, considered substantial to almost-perfect reproducibility. Secondary objectives were to assess which method had highest observed ICC and to assess whether observers identified the same “hot-spots”.

Results: ICC for the global method was 0.87 (95%CI: 0.799-0.93), marginally meeting the prespecified success criterion. The ICC for the hot-spot method was 0.83 (95%CI: 0.74-0.90) and had a CI extending below the success criterion. Across the 23 labs, geometric mean value of the 30 scores ranged from 8.5 to 19.6 for the global method and from 12.8 to 30.3 for the hot-spot method. The overall mean (95% CI) of these values was 12.9 (11.9-14.0) and 20.9 (19.1-22.8), respectively. Visually, between-laboratory agreement in location of selected hot-spot varies between cases. The median times for scoring were 9 and 6 minutes for global and hot-spot methods respectively.

Conclusions: The global method marginally met the prespecified criterion of success; it should now be evaluated for clinical validity in appropriate cohorts of cases. The hot-spot method was observed to have slightly less reproducibility between labs. The time taken for scoring by either method is practical using counting software we are making publicly available. Establishment of external quality assessment schemes is likely to improve the reproducibility between labs further.

(Supported by a grant from the Breast Cancer Research Foundation)

Citation Format: Nielsen TO, Leung SCY, Zabaglo LA, Arun I, Badve SS, Bane AL, Bartlet JMS, Borgquist S, Chang MC, Dodson A, Ehinger A, Fineberg S, Focke CM, Gao D, Gown AM, Gutierrez C, Hugh JC, Kos Z, Lænkholm A-V, Mastropasqua MG, Moriya T, Nofech-Mozes S, Osborne CK, Penault-Llorca FM, Piper T, Sakatani T, Salgado R, Starczynski J, Sugie T, van der Vegt B, Viale G, Hayes DF, McShane LM, Dowsett M. Analytical validation of a standardized scoring protocol for Ki67 assessed on breast excision whole sections: An international multicenter collaboration [abstract]. In: Proceedings of the 2017 San Antonio Breast Cancer Symposium; 2017 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2018;78(4 Suppl):Abstract nr P2-03-01.

An international Ki67 reproducibility study

BACKGROUND

In breast cancer, immunohistochemical assessment of proliferation using the marker Ki67 has potential use in both research and clinical management. However, lack of consistency across laboratories has limited Ki67's value. A working group was assembled to devise a strategy to harmonize Ki67 analysis and increase scoring concordance. Toward that goal, we conducted a Ki67 reproducibility study.

METHODS

Eight laboratories received 100 breast cancer cases arranged into 1-mm core tissue microarrays-one set stained by the participating laboratory and one set stained by the central laboratory, both using antibody MIB-1. Each laboratory scored Ki67 as percentage of positively stained invasive tumor cells using its own method. Six laboratories repeated scoring of 50 locally stained cases on 3 different days. Sources of variation were analyzed using random effects models with log2-transformed measurements. Reproducibility was quantified by intraclass correlation coefficient (ICC), and the approximate two-sided 95% confidence intervals (CIs) for the true intraclass correlation coefficients in these experiments were provided.

RESULTS

Intralaboratory reproducibility was high (ICC = 0.94; 95% CI = 0.93 to 0.97). Interlaboratory reproducibility was only moderate (central staining: ICC = 0.71, 95% CI = 0.47 to 0.78; local staining: ICC = 0.59, 95% CI = 0.37 to 0.68). Geometric mean of Ki67 values for each laboratory across the 100 cases ranged 7.1% to 23.9% with central staining and 6.1% to 30.1% with local staining. Factors contributing to interlaboratory discordance included tumor region selection, counting method, and subjective assessment of staining positivity. Formal counting methods gave more consistent results than visual estimation.

CONCLUSIONS

Substantial variability in Ki67 scoring was observed among some of the world's most experienced laboratories. Ki67 values and cutoffs for clinical decision-making cannot be transferred between laboratories without standardizing scoring methodology because analytical validity is limited.