Format

Send to

Choose Destination
PLoS Biol. 2019 Jan 29;17(1):e3000125. doi: 10.1371/journal.pbio.3000125. eCollection 2019 Jan.

Developing a modern data workflow for regularly updated data.

Author information

1
Department of Wildlife Ecology and Conservation, University of Florida, Gainesville, Florida, United States of America.
2
School of Natural Resources and the Environment, University of Florida, Gainesville, Florida, United States of America.
3
Data Analytics Program, Denison University, Granville, Ohio, United States of America.
4
Informatics Institute, University of Florida, Gainesville, Florida, United States of America.
5
Biodiversity Institute, University of Florida, Gainesville, Florida, United States of America.

Abstract

Over the past decade, biology has undergone a data revolution in how researchers collect data and the amount of data being collected. An emerging challenge that has received limited attention in biology is managing, working with, and providing access to data under continual active collection. Regularly updated data present unique challenges in quality assurance and control, data publication, archiving, and reproducibility. We developed a workflow for a long-term ecological study that addresses many of the challenges associated with managing this type of data. We do this by leveraging existing tools to 1) perform quality assurance and control; 2) import, restructure, version, and archive data; 3) rapidly publish new data in ways that ensure appropriate credit to all contributors; and 4) automate most steps in the data pipeline to reduce the time and effort required by researchers. The workflow leverages tools from software development, including version control and continuous integration, to create a modern data management system that automates the pipeline.

Supplemental Content

Full text links

Icon for Public Library of Science Icon for PubMed Central
Loading ...
Support Center