Format

Send to

Choose Destination
Proc Natl Acad Sci U S A. 2017 Nov 28;114(48):12708-12713. doi: 10.1073/pnas.1707323114. Epub 2017 Nov 14.

Reviewer bias in single- versus double-blind peer review.

Author information

1
Google, Inc., Mountain View, CA 94043; atomkins@gmail.com.
2
State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China.
3
Google, Inc., Mountain View, CA 94043.

Abstract

Peer review may be "single-blind," in which reviewers are aware of the names and affiliations of paper authors, or "double-blind," in which this information is hidden. Noting that computer science research often appears first or exclusively in peer-reviewed conferences rather than journals, we study these two reviewing models in the context of the 10th Association for Computing Machinery International Conference on Web Search and Data Mining, a highly selective venue (15.6% acceptance rate) in which expert committee members review full-length submissions for acceptance. We present a controlled experiment in which four committee members review each paper. Two of these four reviewers are drawn from a pool of committee members with access to author information; the other two are drawn from a disjoint pool without such access. This information asymmetry persists through the process of bidding for papers, reviewing papers, and entering scores. Reviewers in the single-blind condition typically bid for 22% fewer papers and preferentially bid for papers from top universities and companies. Once papers are allocated to reviewers, single-blind reviewers are significantly more likely than their double-blind counterparts to recommend for acceptance papers from famous authors, top universities, and top companies. The estimated odds multipliers are tangible, at 1.63, 1.58, and 2.10, respectively.

KEYWORDS:

double-blind; peer review; scientific method

PMID:
29138317
PMCID:
PMC5715744
DOI:
10.1073/pnas.1707323114
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for HighWire Icon for PubMed Central
Loading ...
Support Center