Format

Send to

Choose Destination
Gigascience. 2018 Jan 1;7(1):1-6. doi: 10.1093/gigascience/gix120.

SOAPnuke: a MapReduce acceleration-supported software for integrated quality control and preprocessing of high-throughput sequencing data.

Chen Y1, Chen Y2, Shi C3,4,5, Huang Z1, Zhang Y1,6, Li S1,6, Li Y1, Ye J1, Yu C7, Li Z8,9, Zhang X1, Wang J1,10, Yang H1,10, Fang L1,6, Chen Q3,4,5.

Author information

1
BGI-Shenzhen, Shenzhen 518083.
2
Geneplus-Beijing, Beijing 102206.
3
Department of Oncology, Fujian Medical University Union Hospital, Fuzhou 350001.
4
Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou 350014.
5
Department of Stem Cell Research Institute, Fujian Medical University Stem Cell Research Institute, Fuzhou 350000.
6
Collaborative Innovation Center of High Performance Computing, National University of Defense Technology, Changsha 410073.
7
Intel China Ltd., Shanghai 200336.
8
Guangdong Provincial Hospital of Chinese Medicine, Guangzhou 510120.
9
Department of Surgery, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong.
10
James D. Watson Institute of Genome Sciences, Hangzhou 310058, China.

Abstract

Quality control (QC) and preprocessing are essential steps for sequencing data analysis to ensure the accuracy of results. However, existing tools cannot provide a satisfying solution with integrated comprehensive functions, proper architectures, and highly scalable acceleration. In this article, we demonstrate SOAPnuke as a tool with abundant functions for a "QC-Preprocess-QC" workflow and MapReduce acceleration framework. Four modules with different preprocessing functions are designed for processing datasets from genomic, small RNA, Digital Gene Expression, and metagenomic experiments, respectively. As a workflow-like tool, SOAPnuke centralizes processing functions into 1 executable and predefines their order to avoid the necessity of reformatting different files when switching tools. Furthermore, the MapReduce framework enables large scalability to distribute all the processing works to an entire compute cluster.We conducted a benchmarking where SOAPnuke and other tools are used to preprocess a ∼30× NA12878 dataset published by GIAB. The standalone operation of SOAPnuke struck a balance between resource occupancy and performance. When accelerated on 16 working nodes with MapReduce, SOAPnuke achieved ∼5.7 times the fastest speed of other tools.

KEYWORDS:

MapReduce; high-throughput sequencing; preprocessing; quality control

PMID:
29220494
PMCID:
PMC5788068
DOI:
10.1093/gigascience/gix120
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Silverchair Information Systems Icon for PubMed Central
Loading ...
Support Center