Display Settings:

Format

Send to:

Choose Destination
See comment in PubMed Commons below
PLoS One. 2013 Apr 24;8(4):e61562. doi: 10.1371/journal.pone.0061562. Print 2013.

A plea for neutral comparison studies in computational sciences.

Author information

  • 1Department of Medical Informatics, Biometry and Epidemiology, Ludwig-Maximilians-University of Munich, Munich, Germany. boulesteix@ibe.med.uni-muenchen.de

Abstract

In computational science literature including, e.g., bioinformatics, computational statistics or machine learning, most published articles are devoted to the development of "new methods", while comparison studies are generally appreciated by readers but surprisingly given poor consideration by many journals. This paper stresses the importance of neutral comparison studies for the objective evaluation of existing methods and the establishment of standards by drawing parallels with clinical research. The goal of the paper is twofold. Firstly, we present a survey of recent computational papers on supervised classification published in seven high-ranking computational science journals. The aim is to provide an up-to-date picture of current scientific practice with respect to the comparison of methods in both articles presenting new methods and articles focusing on the comparison study itself. Secondly, based on the results of our survey we critically discuss the necessity, impact and limitations of neutral comparison studies in computational sciences. We define three reasonable criteria a comparison study has to fulfill in order to be considered as neutral, and explicate general considerations on the individual components of a "tidy neutral comparison study". R codes for completely replicating our statistical analyses and figures are available from the companion website http://www.ibe.med.uni-muenchen.de/organisation/mitarbeiter/020_professuren/boulesteix/plea2013.

PMID:
23637855
[PubMed - indexed for MEDLINE]
PMCID:
PMC3634809
Free PMC Article

Images from this publication.See all images (4)Free text

Figure 1
Figure 2
Figure 3
Figure 4
PubMed Commons home

PubMed Commons

0 comments
How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for Public Library of Science Icon for PubMed Central
    Loading ...
    Write to the Help Desk