Evaluating the analytical validity of circulating tumor DNA sequencing assays for precision oncology

Circulating tumor DNA (ctDNA) sequencing is being rapidly adopted in precision oncology, but the accuracy, sensitivity and reproducibility of ctDNA assays is poorly understood. Here we report the findings of a multi-site, cross-platform evaluation of the analytical performance of five industry-leading ctDNA assays. We evaluated each stage of the ctDNA sequencing workflow with simulations, synthetic DNA spike-in experiments and proficiency testing on standardized, cell-line-derived reference samples. Above 0.5% variant allele frequency, ctDNA mutations were detected with high sensitivity, precision and reproducibility by all five assays, whereas, below this limit, detection became unreliable and varied widely between assays, especially when input material was limited. Missed mutations (false negatives) were more common than erroneous candidates (false positives), indicating that the reliable sampling of rare ctDNA fragments is the key challenge for ctDNA assays. This comprehensive evaluation of the analytical performance of ctDNA assays serves to inform best practice guidelines and provides a resource for precision oncology.

Cross-oncopanel study reveals high sensitivity and accuracy with overall analytical performance depending on genomic regions

BACKGROUND

Targeted sequencing using oncopanels requires comprehensive assessments of accuracy and detection sensitivity to ensure analytical validity. By employing reference materials characterized by the U.S. Food and Drug Administration-led SEquence Quality Control project phase2 (SEQC2) effort, we perform a cross-platform multi-lab evaluation of eight Pan-Cancer panels to assess best practices for oncopanel sequencing.

RESULTS

All panels demonstrate high sensitivity across targeted high-confidence coding regions and variant types for the variants previously verified to have variant allele frequency (VAF) in the 5-20% range. Sensitivity is reduced by utilizing VAF thresholds due to inherent variability in VAF measurements. Enforcing a VAF threshold for reporting has a positive impact on reducing false positive calls. Importantly, the false positive rate is found to be significantly higher outside the high-confidence coding regions, resulting in lower reproducibility. Thus, region restriction and VAF thresholds lead to low relative technical variability in estimating promising biomarkers and tumor mutational burden.

CONCLUSION

This comprehensive study provides actionable guidelines for oncopanel sequencing and clear evidence that supports a simplified approach to assess the analytical performance of oncopanels. It will facilitate the rapid implementation, validation, and quality control of oncopanels in clinical use.

A verified genomic reference sample for assessing performance of cancer panels detecting small variants of low allele frequency

BACKGROUND

Oncopanel genomic testing, which identifies important somatic variants, is increasingly common in medical practice and especially in clinical trials. Currently, there is a paucity of reliable genomic reference samples having a suitably large number of pre-identified variants for properly assessing oncopanel assay analytical quality and performance. The FDA-led Sequencing and Quality Control Phase 2 (SEQC2) consortium analyze ten diverse cancer cell lines individually and their pool, termed Sample A, to develop a reference sample with suitably large numbers of coding positions with known (variant) positives and negatives for properly evaluating oncopanel analytical performance.

RESULTS

In reference Sample A, we identify more than 40,000 variants down to 1% allele frequency with more than 25,000 variants having less than 20% allele frequency with 1653 variants in COSMIC-related genes. This is 5-100× more than existing commercially available samples. We also identify an unprecedented number of negative positions in coding regions, allowing statistical rigor in assessing limit-of-detection, sensitivity, and precision. Over 300 loci are randomly selected and independently verified via droplet digital PCR with 100% concordance. Agilent normal reference Sample B can be admixed with Sample A to create new samples with a similar number of known variants at much lower allele frequency than what exists in Sample A natively, including known variants having allele frequency of 0.02%, a range suitable for assessing liquid biopsy panels.

CONCLUSION

These new reference samples and their admixtures provide superior capability for performing oncopanel quality control, analytical accuracy, and validation for small to large oncopanels and liquid biopsy assays.

MammaPrint and BluePrint Molecular Diagnostics Using Targeted RNA Next-Generation Sequencing Technology

Next-generation DNA sequencing is rapidly becoming an indispensable tool for genome-directed cancer diagnostics, but next-generation RNA sequencing (RNA-seq) is currently not standardly used in clinical diagnostics for expression assessment. However, multigene RNA diagnostic assays are used increasingly in the routine diagnosis of early-stage breast cancer. Two of the most widely used tests are currently available only as a central laboratory service, which limits their clinical use. We evaluated the use of RNA-seq as a decentralized method to perform such tests. The MammaPrint and BluePrint RNA-seq tests were found to be equivalent to the clinically validated microarray tests. The RNA-seq tests were highly reproducible when performed in different locations and were stable over time. The MammaPrint RNA-seq test was clinically validated. Our data demonstrate that RNA-seq can be used as a decentralized platform, yielding results substantially equivalent to results derived from the predicate diagnostic device.

Enrichment of sequencing targets from the human genome by solution hybridization

A method for target sequence enrichment from the human genome is described. This hybridization-based approach using oligonucleotide probes in solution has excellent sensitivity and accuracy for calling SNPs

To exploit fully the potential of current sequencing technologies for population-based studies, one must enrich for loci from the human genome. Here we evaluate the hybridization-based approach by using oligonucleotide capture probes in solution to enrich for approximately 3.9 Mb of sequence target. We demonstrate that the tiling probe frequency is important for generating sequence data with high uniform coverage of targets. We obtained 93% sensitivity to detect SNPs, with a calling accuracy greater than 99%.

Characterization of mutations in ATP8B1 associated with hereditary cholestasis

Progressive familial intrahepatic cholestasis (PFIC) and benign recurrent intrahepatic cholestasis (BRIC) are clinically distinct hereditary disorders. PFIC patients suffer from chronic cholestasis and develop liver fibrosis. BRIC patients experience intermittent attacks of cholestasis that resolve spontaneously. Mutations in ATP8B1 (previously FIC1) may result in PFIC or BRIC. We report the genomic organization of ATP8B1 and mutation analyses of 180 families with PFIC or BRIC that identified 54 distinct disease mutations, including 10 mutations predicted to disrupt splicing, 6 nonsense mutations, 11 small insertion or deletion mutations predicted to induce frameshifts, 1 large genomic deletion, 2 small inframe deletions, and 24 missense mutations. Most mutations are rare, occurring in 1-3 families, or are limited to specific populations. Many patients are compound heterozygous for 2 mutations. Mutation type or location correlates overall with clinical severity: missense mutations are more common in BRIC (58% vs. 38% in PFIC), while nonsense, frameshifting, and large deletion mutations are more common in PFIC (41% vs. 16% in BRIC). Some mutations, however, lead to a wide range of phenotypes, from PFIC to BRIC or even no clinical disease. ATP8B1 mutations were detected in 30% and 41%, respectively, of the PFIC and BRIC patients screened.