Format

Send to

Choose Destination
Genome Biol. 2018 Nov 6;19(1):188. doi: 10.1186/s13059-018-1539-5.

Combining accurate tumor genome simulation with crowdsourcing to benchmark somatic structural variant detection.

Author information

1
Ontario Institute for Cancer Research, Toronto, Ontario, Canada.
2
Department of Biomolecular Engineering, University of California, Santa Cruz, Santa Cruz, CA, USA.
3
Mater Research Institute, University of Queensland, Woolloongabba, QLD, Australia.
4
Computational Biology Program, Oregon Health & Science University, Portland, OR, USA.
5
Sage Bionetworks, Seattle, WA, USA.
6
Department of Bioinformatics and Computational Biology, University of Texas MD Anderson Cancer Center, Houston, TX, USA.
7
Department of Genetics, University of Alabama at Birmingham, Birmingham, AL, USA.
8
Informatics Institute, University of Alabama at Birmingham, Birmingham, AL, USA.
9
IBM Computational Biology Center, T.J.Watson Research Center, Yorktown Heights, NY, USA.
10
Computational Biology Program, Oregon Health & Science University, Portland, OR, USA. adam.margolin@mssm.edu.
11
Sage Bionetworks, Seattle, WA, USA. adam.margolin@mssm.edu.
12
Department of Biomolecular Engineering, University of California, Santa Cruz, Santa Cruz, CA, USA. jstuart@ucsc.edu.
13
Ontario Institute for Cancer Research, Toronto, Ontario, Canada. paul.boutros@oicr.on.ca.
14
Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada. paul.boutros@oicr.on.ca.
15
Department of Pharmacology and Toxicology, University of Toronto, Toronto, Ontario, Canada. paul.boutros@oicr.on.ca.

Abstract

BACKGROUND:

The phenotypes of cancer cells are driven in part by somatic structural variants. Structural variants can initiate tumors, enhance their aggressiveness, and provide unique therapeutic opportunities. Whole-genome sequencing of tumors can allow exhaustive identification of the specific structural variants present in an individual cancer, facilitating both clinical diagnostics and the discovery of novel mutagenic mechanisms. A plethora of somatic structural variant detection algorithms have been created to enable these discoveries; however, there are no systematic benchmarks of them. Rigorous performance evaluation of somatic structural variant detection methods has been challenged by the lack of gold standards, extensive resource requirements, and difficulties arising from the need to share personal genomic information.

RESULTS:

To facilitate structural variant detection algorithm evaluations, we create a robust simulation framework for somatic structural variants by extending the BAMSurgeon algorithm. We then organize and enable a crowdsourced benchmarking within the ICGC-TCGA DREAM Somatic Mutation Calling Challenge (SMC-DNA). We report here the results of structural variant benchmarking on three different tumors, comprising 204 submissions from 15 teams. In addition to ranking methods, we identify characteristic error profiles of individual algorithms and general trends across them. Surprisingly, we find that ensembles of analysis pipelines do not always outperform the best individual method, indicating a need for new ways to aggregate somatic structural variant detection approaches.

CONCLUSIONS:

The synthetic tumors and somatic structural variant detection leaderboards remain available as a community benchmarking resource, and BAMSurgeon is available at https://github.com/adamewing/bamsurgeon .

KEYWORDS:

Benchmarking; Cancer genomics; Crowdsourcing; Simulation; Somatic mutations; Structural variants; Whole-genome sequencing

Supplemental Content

Full text links

Icon for BioMed Central Icon for PubMed Central
Loading ...
Support Center