3
|
Deveson IW, Gong B, Lai K, LoCoco JS, Richmond TA, Schageman J, Zhang Z, Novoradovskaya N, Willey JC, Jones W, Kusko R, Chen G, Madala BS, Blackburn J, Stevanovski I, Bhandari A, Close D, Conroy J, Hubank M, Marella N, Mieczkowski PA, Qiu F, Sebra R, Stetson D, Sun L, Szankasi P, Tan H, Tang LY, Arib H, Best H, Burgher B, Bushel PR, Casey F, Cawley S, Chang CJ, Choi J, Dinis J, Duncan D, Eterovic AK, Feng L, Ghosal A, Giorda K, Glenn S, Happe S, Haseley N, Horvath K, Hung LY, Jarosz M, Kushwaha G, Li D, Li QZ, Li Z, Liu LC, Liu Z, Ma C, Mason CE, Megherbi DB, Morrison T, Pabón-Peña C, Pirooznia M, Proszek PZ, Raymond A, Rindler P, Ringler R, Scherer A, Shaknovich R, Shi T, Smith M, Song P, Strahl M, Thodima VJ, Tom N, Verma S, Wang J, Wu L, Xiao W, Xu C, Yang M, Zhang G, Zhang S, Zhang Y, Shi L, Tong W, Johann DJ, Mercer TR, Xu J. Evaluating the analytical validity of circulating tumor DNA sequencing assays for precision oncology. Nat Biotechnol 2021; 39:1115-1128. [PMID: 33846644 PMCID: PMC8434938 DOI: 10.1038/s41587-021-00857-z] [Citation(s) in RCA: 135] [Impact Index Per Article: 33.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2020] [Accepted: 02/15/2021] [Indexed: 02/08/2023]
Abstract
Circulating tumor DNA (ctDNA) sequencing is being rapidly adopted in precision oncology, but the accuracy, sensitivity and reproducibility of ctDNA assays is poorly understood. Here we report the findings of a multi-site, cross-platform evaluation of the analytical performance of five industry-leading ctDNA assays. We evaluated each stage of the ctDNA sequencing workflow with simulations, synthetic DNA spike-in experiments and proficiency testing on standardized, cell-line-derived reference samples. Above 0.5% variant allele frequency, ctDNA mutations were detected with high sensitivity, precision and reproducibility by all five assays, whereas, below this limit, detection became unreliable and varied widely between assays, especially when input material was limited. Missed mutations (false negatives) were more common than erroneous candidates (false positives), indicating that the reliable sampling of rare ctDNA fragments is the key challenge for ctDNA assays. This comprehensive evaluation of the analytical performance of ctDNA assays serves to inform best practice guidelines and provides a resource for precision oncology.
Collapse
Affiliation(s)
- Ira W Deveson
- Kinghorn Centre for Clinical Genomics, Garvan Institute of Medical Research, Sydney, NSW, Australia
- St. Vincent's Clinical School, Faculty of Medicine, University of New South Wales, Sydney, NSW, Australia
| | - Binsheng Gong
- Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, US Food and Drug Administration, Jefferson, AR, USA
| | - Kevin Lai
- Bioinformatics, Integrated DNA Technologies, Inc., Coralville, IA, USA
| | | | - Todd A Richmond
- Market & Application Development Bioinformatics, Roche Sequencing Solutions Inc., Pleasanton, CA, USA
| | | | - Zhihong Zhang
- Research and Development, Burning Rock Biotech, Shanghai, China
| | | | - James C Willey
- Departments of Medicine, Pathology, and Cancer Biology, College of Medicine and Life Sciences, University of Toledo Health Sciences Campus, Toledo, OH, USA
| | | | | | - Guangchun Chen
- Department of Immunology, Genomics and Microarray Core Facility, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Bindu Swapna Madala
- Genomics and Epigenetics Theme, Garvan Institute of Medical Research, Sydney, NSW, Australia
| | - James Blackburn
- Cancer Theme, Garvan Institute of Medical Research, Sydney, NSW, Australia
- St. Vincent's Clinical School, University of New South Wales, Sydney, NSW, Australia
| | - Igor Stevanovski
- Kinghorn Centre for Clinical Genomics, Garvan Institute of Medical Research, Sydney, NSW, Australia
| | | | - Devin Close
- R&D Genomics MPS, Institute for Clinical and Experimental Pathology ARUP Laboratories, Salt Lake City, UT, USA
| | | | - Michael Hubank
- NIHR Biomedical Research Centre, Royal Marsden Hospital, Sutton, Surrey, UK
| | | | | | - Fujun Qiu
- Research and Development, Burning Rock Biotech, Shanghai, China
| | - Robert Sebra
- Icahn Institute and Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | | | - Lihyun Sun
- Elim Biopharmaceuticals, Inc., Hayward, CA, USA
| | - Philippe Szankasi
- R&D Genomics MPS, Institute for Clinical and Experimental Pathology ARUP Laboratories, Salt Lake City, UT, USA
| | - Haowen Tan
- Primbio Genes Biotechnology, East Lake High-tech Development Zone, Wuhan, Hubei, China
| | - Lin-Ya Tang
- Institute for Personalized Cancer Therapy, MD Anderson Cancer Center, Houston, TX, USA
| | - Hanane Arib
- Icahn Institute and Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Hunter Best
- R&D Genomics MPS, Institute for Clinical and Experimental Pathology ARUP Laboratories, Salt Lake City, UT, USA
- Departments of Pathology and Pediatrics, University of Utah School of Medicine, Salt Lake City, UT, USA
| | | | - Pierre R Bushel
- National Institute of Environmental Health Sciences, Research Triangle Park, Morrisville, NC, USA
| | - Fergal Casey
- Market & Application Development Bioinformatics, Roche Sequencing Solutions Inc., Pleasanton, CA, USA
| | - Simon Cawley
- Clinical Sequencing Division, Thermo Fisher Scientific, South San Francisco, CA, USA
| | - Chia-Jung Chang
- Stanford Genome Technology Center, Stanford University, Palo Alto, CA, USA
| | - Jonathan Choi
- Roche Sequencing Solutions, Inc., Pleasanton, CA, USA
| | - Jorge Dinis
- Roche Sequencing Solutions, Inc., Pleasanton, CA, USA
| | | | - Agda Karina Eterovic
- Institute for Personalized Cancer Therapy, MD Anderson Cancer Center, Houston, TX, USA
| | - Liang Feng
- Market & Application Development Bioinformatics, Roche Sequencing Solutions Inc., Pleasanton, CA, USA
| | | | - Kristina Giorda
- Marketing, Integrated DNA Technologies, Inc., Coralville, IA, USA
| | | | | | | | | | - Li-Yuan Hung
- Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Mirna Jarosz
- NGS Products and Services, Integrated DNA Technologies, Inc., Coralville, IA, USA
| | - Garima Kushwaha
- Market & Application Development Bioinformatics, Roche Sequencing Solutions Inc., Pleasanton, CA, USA
| | - Dan Li
- Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, US Food and Drug Administration, Jefferson, AR, USA
| | - Quan-Zhen Li
- Department of Immunology, Genomics and Microarray Core Facility, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Zhiguang Li
- Intramural Research Program, Laboratory of Epidemiology and Population Sciences, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
| | - Liang-Chun Liu
- Clinical Diagnostic Division, Thermo Fisher Scientific, Fremont, CA, USA
| | - Zhichao Liu
- Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, US Food and Drug Administration, Jefferson, AR, USA
| | - Charles Ma
- Cancer Genetics, Inc., Rutherford, NJ, USA
| | - Christopher E Mason
- Department of Physiology and Biophysics, Weill Cornell Medicine, Cornell University, New York, NY, USA
| | - Dalila B Megherbi
- CMINDS Research Center, Department of Electrical and Computer Engineering, College of Engineering, University of Massachusetts Lowell, Lowell, MA, USA
| | | | | | - Mehdi Pirooznia
- Bioinformatics and Computational Biology Laboratory, National Heart Lung and Blood Institute, National Institutes of Health, Bethesda, MD, USA
| | - Paula Z Proszek
- NIHR Biomedical Research Centre, Royal Marsden Hospital, Sutton, Surrey, UK
| | | | - Paul Rindler
- R&D Genomics MPS, Institute for Clinical and Experimental Pathology ARUP Laboratories, Salt Lake City, UT, USA
| | | | - Andreas Scherer
- Institute for Molecular Medicine Finland (FIMM), Nordic EMBL Partnership for Molecular Medicine, HiLIFE Unit, Biomedicum Helsinki 2U (D302b), University of Helsinki, Helsinki, Finland
- EATRIS ERIC- European Infrastructure for Translational Medicine, Amsterdam, The Netherlands
| | | | - Tieliu Shi
- Center for Bioinformatics and Computational Biology and the Institute of Biomedical Sciences, School of Life Sciences, East China Normal University, Shanghai, China
| | - Melissa Smith
- Icahn Institute and Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Ping Song
- Institute for Personalized Cancer Therapy, MD Anderson Cancer Center, Houston, TX, USA
| | - Maya Strahl
- Icahn Institute and Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | | | - Nikola Tom
- EATRIS ERIC- European Infrastructure for Translational Medicine, Amsterdam, The Netherlands
- Center of Molecular Medicine, Central European Institute of Technology, Masaryk University, Brno, Czech Republic
| | | | - Jiashi Wang
- Research and Development, Integrated DNA Technologies, Inc., Coralville, IA, USA
| | - Leihong Wu
- Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, US Food and Drug Administration, Jefferson, AR, USA
| | - Wenzhong Xiao
- Stanford Genome Technology Center, Stanford University, Palo Alto, CA, USA
- Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Chang Xu
- Research and Development, QIAGEN Sciences, Inc., Frederick, MD, USA
| | - Mary Yang
- Department of Information Science, University of Arkansas at Little Rock, Little Rock, AR, USA
| | | | - Sa Zhang
- Clinical Laboratory, Burning Rock Biotech, Guangzhou, China
| | - Yilin Zhang
- Elim Biopharmaceuticals, Inc., Hayward, CA, USA
| | - Leming Shi
- State Key Laboratory of Genetic Engineering, School of Life Sciences and Shanghai Cancer Hospital/Cancer Institute, Fudan University, Shanghai, China
- Human Phenome Institute, Fudan University, Shanghai, China
- Fudan-Gospel Joint Research Center for Precision Medicine, Fudan University, Shanghai, China
| | - Weida Tong
- Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, US Food and Drug Administration, Jefferson, AR, USA
| | - Donald J Johann
- Winthrop P. Rockefeller Cancer Institute, University of Arkansas for Medical Sciences, Little Rock, AR, USA.
| | - Timothy R Mercer
- St. Vincent's Clinical School, Faculty of Medicine, University of New South Wales, Sydney, NSW, Australia.
- Genomics and Epigenetics Theme, Garvan Institute of Medical Research, Sydney, NSW, Australia.
- Australian Institute of Bioengineering and Nanotechnology, University of Queensland, Queensland, QLD, Australia.
| | - Joshua Xu
- Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, US Food and Drug Administration, Jefferson, AR, USA.
| |
Collapse
|