TESTING ERRORS WITH LIQUID BIOPSIES

Noninvasive genotyping of plasma cell-free DNA is being integrated into cancer care at an astonishing rate, as its potential to radically increase access to personalized cancer care is increasingly recognized.^[1] Whereas next-generation sequencing (NGS) of tumor has been commoditized and democratized (with many academic hospitals offering their own Clinical Laboratory Improvement Amendments–certified tests), plasma NGS is primarily sent out for testing at commercial laboratories. This is in part because of the ease of shipping a tube of blood and in part because of the technological advances required to confidently perform plasma NGS. However, although these tests are ordered and reported en masse, the causes of false-positive or false-negative plasma NGS results have only gradually become apparent (Figure 1).

Figure 1.

Common sources of false-positive and false-negative results in plasma next-generation sequencing (NGS). False-negative results have largely been associated with low tumor DNA shed below an assay's technical limit of detection. False-positive results have been attributed to genomic heterogeneity of the tumor, particularly in the setting of acquired drug resistance. Both germline and noncancerous somatic variants (eg, clonal hematopoiesis) are also known to confound the analysis and interpretation of plasma NGS results. Beyond these established biologic factors, technical factors may be an underappreciated source of erroneous plasma NGS.

False-negative results have long been recognized as common with blood-based assays. We and others identified that assay sensitivity is closely related to clinical factors such as stage and metastatic spread, suggesting limited shed of tumor DNA in cases missed by plasma NGS.^[2,3] False positives have also been described with many plasma genotyping assays and are routinely attributed to tumor heterogeneity.^[4] Although heterogeneity is clearly a source of tumor/plasma discordance in cancers that have developed drug resistance,^[5,6]many false positives can be attributed to DNA shed from normal cells, including germline variants or noncancerous somatic variants such as clonal hematopoiesis (CH).^[7–9] The latter is particularly challenging because CH can involve cancer-associated genes (eg, TP53, JAK2, KRAS). Yet when such mutations are identified in plasma, they may not reflect true tumor genotype. What is least understood is how technical factors related to assay performance contribute to false-positive and false-negative results. Probe design and specifics of library generation are often considered proprietary and are only minimally described in the few analytical validation studies that have been published.

This is the context motivating the publication by Stetson et al^[10] in JCO Precision Oncology. The authors sent aliquots of plasma to four commercial Clinical Laboratory Improvement Amendments–certified laboratories for plasma NGS and compared this to NGS performed on matched tumor and normal tissue at Foundation Medicine. Plasma was collected 1 to 4 hours before treatment or surgery and sent as 2-mL aliquots for retrospective NGS testing. The vendors were blinded to tumor NGS results but knew that their plasma NGS results would be compared with orthogonal tissue results, as well as with plasma results from other laboratories. Following bioinformatic analysis and variant calling, binary alignment map files were provided by the vendors to the authors for independent, unblinded, post-hoc sequence analysis. With this design, the authors were not only able to address many of the pitfalls that plagued prior vendor comparison studies,^[11,12] but they could also investigate interassay technical factors. More precisely, Stetson et al^[10] were able to account for known germline variants, investigate mutation-calling biases, and examine whether false negatives were as the result of stochastic sample biases or thresholding nuances of the vendor's bioinformatic filter.

The results should give pause. Positive predictive value (PPV) against tissue ranged from 36% to 80%. However, results improved when limited to mutations called at an allelic fraction (AF) greater than 1%, with three vendors achieving a PPV of 100% for these higher AF calls. Focusing on mutations called with an AF less than 1%, PPVs were as low as 17%. Note that variants detected at less than 1% AF are routinely reported by each vendor, and such sensitivity is advertised as a unique strength of plasma NGS assays. False-positive variants tended to be novel with no reports in somatic variant databases, and these were often related to vendor-specific mutational biases. False negatives were attributed to bioinformatic filtering of suspected germline variants and limitations from high signal-to-noise ratio. Taken together, these results point to recurrent contributions from technical differences in the bioinformatic pipelines of the assays, assay sensitivity, or plain error.

The current study is far from perfect and leaves room for improvement. The study cohort consisted primarily of early-stage cancers (21 of 24 patients had stage I and II cancers), which are not the intended population for plasma NGS. This may result in a stochastic bias when comparing 2-mL aliquots because of low tumor DNA content. The study also focused predominantly on nondriver single–base pair mutations with few complex variants (eg, fusions, indels). This made it difficult to extrapolate to clinical NGS performed for detection of targetable mutations, indels, and fusions. Finally, reference truth was extrapolated across four different assays. Indeed, even years after the first commercial plasma NGS test entered the market, we still are not sure what truth is in plasma. Furthermore, this analysis does not broach the challenges of reporting plasma NGS results. One could imagine that a forward-thinking laboratory might recognize the limitations of plasma NGS in its reporting, noting the risk of false positives for low AF or CH-associated variants, as well as the risk of false negatives when low tumor shed is apparent.

As the use of plasma NGS becomes increasingly widespread in cancer care, there remains a clear need for concordance studies such as this one. Future studies would ideally focus on actionable variants and would be limited to advanced cancer. In addition, we have found that orthogonal benchmarking against an established assay, such as digital droplet polymerase chain reaction, is a powerful way to establish a reference point for such analyses.^[13,14]Whereas such multisite proficiency testing is common for tumor genotyping (for which large specimens are often available and can be divided), plasma cell-free DNA can be a scarce specimen, which makes paired analysis across multiple laboratories extremely challenging. Additional investment in validated reference materials could be one step toward establishing a reference point that laboratories can use to confirm the accuracy of their results, as well as a step toward improving the quality of testing for our patients with cancer.