Format

Send to

Choose Destination
Nat Biotechnol. 2019 Apr 1. doi: 10.1038/s41587-019-0074-6. [Epub ahead of print]

An open resource for accurately benchmarking small variant and reference calls.

Author information

1
Material Measurement Laboratory, National Institute of Standards and Technology, Gaithersburg, MD, USA. jzook@nist.gov.
2
Material Measurement Laboratory, National Institute of Standards and Technology, Gaithersburg, MD, USA.
3
10x Genomics, Pleasanton, CA, USA.
4
Wellcome Trust Sanger Institute,, Hinxton, Cambridge, UK.
5
Real Time Genomics, Hamilton, New Zealand.
6
Invitae Corporation, San Francisco, CA, USA.
7
Verily Life Sciences, South San Francisco, CA, USA.
8
Google Inc., Mountain View, CA, USA.
9
Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA.
10
National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, MD, USA.
11
Joint Initiative for Metrology in Biology, Stanford, CA, USA.
12
Department of Bioengineering, Stanford University, Stanford, CA, USA.

Abstract

Benchmark small variant calls are required for developing, optimizing and assessing the performance of sequencing and bioinformatics methods. Here, as part of the Genome in a Bottle (GIAB) Consortium, we apply a reproducible, cloud-based pipeline to integrate multiple short- and linked-read sequencing datasets and provide benchmark calls for human genomes. We generate benchmark calls for one previously analyzed GIAB sample, as well as six genomes from the Personal Genome Project. These new genomes have broad, open consent, making this a 'first of its kind' resource that is available to the community for multiple downstream applications. We produce 17% more benchmark single nucleotide variations, 176% more indels and 12% larger benchmark regions than previously published GIAB benchmarks. We demonstrate that this benchmark reliably identifies errors in existing callsets and highlight challenges in interpreting performance metrics when using benchmarks that are not perfect or comprehensive. Finally, we identify strengths and weaknesses of callsets by stratifying performance according to variant type and genome context.

PMID:
30936564
DOI:
10.1038/s41587-019-0074-6

Supplemental Content

Full text links

Icon for Nature Publishing Group
Loading ...
Support Center