Format

Send to

Choose Destination
PLoS One. 2015 Sep 3;10(9):e0130433. doi: 10.1371/journal.pone.0130433. eCollection 2015.

A Web Resource for Standardized Benchmark Datasets, Metrics, and Rosetta Protocols for Macromolecular Modeling and Design.

Author information

1
California Institute for Quantitative Biosciences (QB3), University of California San Francisco, San Francisco, California, United States of America; Department of Bioengineering and Therapeutic Sciences, University of California San Francisco, San Francisco, California, United States of America.
2
Graduate Program in Bioinformatics, University of California San Francisco, San Francisco, California, United States of America.
3
Graduate Program in Biophysics, University of California San Francisco, San Francisco, California, United States of America.
4
Department of Pharmaceutical Chemistry, University of California San Francisco, San Francisco, California, United States of America.
5
California Institute for Quantitative Biosciences (QB3), University of California San Francisco, San Francisco, California, United States of America; Department of Bioengineering and Therapeutic Sciences, University of California San Francisco, San Francisco, California, United States of America; Graduate Program in Bioinformatics, University of California San Francisco, San Francisco, California, United States of America.
6
California Institute for Quantitative Biosciences (QB3), University of California San Francisco, San Francisco, California, United States of America; Department of Bioengineering and Therapeutic Sciences, University of California San Francisco, San Francisco, California, United States of America; Graduate Program in Bioinformatics, University of California San Francisco, San Francisco, California, United States of America; Graduate Program in Biophysics, University of California San Francisco, San Francisco, California, United States of America.

Abstract

The development and validation of computational macromolecular modeling and design methods depend on suitable benchmark datasets and informative metrics for comparing protocols. In addition, if a method is intended to be adopted broadly in diverse biological applications, there needs to be information on appropriate parameters for each protocol, as well as metrics describing the expected accuracy compared to experimental data. In certain disciplines, there exist established benchmarks and public resources where experts in a particular methodology are encouraged to supply their most efficient implementation of each particular benchmark. We aim to provide such a resource for protocols in macromolecular modeling and design. We present a freely accessible web resource (https://kortemmelab.ucsf.edu/benchmarks) to guide the development of protocols for protein modeling and design. The site provides benchmark datasets and metrics to compare the performance of a variety of modeling protocols using different computational sampling methods and energy functions, providing a "best practice" set of parameters for each method. Each benchmark has an associated downloadable benchmark capture archive containing the input files, analysis scripts, and tutorials for running the benchmark. The captures may be run with any suitable modeling method; we supply command lines for running the benchmarks using the Rosetta software suite. We have compiled initial benchmarks for the resource spanning three key areas: prediction of energetic effects of mutations, protein design, and protein structure prediction, each with associated state-of-the-art modeling protocols. With the help of the wider macromolecular modeling community, we hope to expand the variety of benchmarks included on the website and continue to evaluate new iterations of current methods as they become available.

PMID:
26335248
PMCID:
PMC4559433
DOI:
10.1371/journal.pone.0130433
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Public Library of Science Icon for PubMed Central
Loading ...
Support Center