Format

Send to

Choose Destination
See comment in PubMed Commons below
PLoS One. 2009 Jun 18;4(6):e5910. doi: 10.1371/journal.pone.0005910.

Looking for landmarks: the role of expert review and bibliometric analysis in evaluating scientific publication outputs.

Author information

1
Wellcome Trust, London, United Kingdom. l.allen@wellcome.ac.uk

Abstract

OBJECTIVE:

To compare expert assessment with bibliometric indicators as tools to assess the quality and importance of scientific research papers.

METHODS AND MATERIALS:

Shortly after their publication in 2005, the quality and importance of a cohort of nearly 700 Wellcome Trust (WT) associated research papers were assessed by expert reviewers; each paper was reviewed by two WT expert reviewers. After 3 years, we compared this initial assessment with other measures of paper impact.

RESULTS:

Shortly after publication, 62 (9%) of the 687 research papers were determined to describe at least a 'major addition to knowledge' -6 were thought to be 'landmark' papers. At an aggregate level, after 3 years, there was a strong positive association between expert assessment and impact as measured by number of citations and F1000 rating. However, there were some important exceptions indicating that bibliometric measures may not be sufficient in isolation as measures of research quality and importance, and especially not for assessing single papers or small groups of research publications.

CONCLUSION:

When attempting to assess the quality and importance of research papers, we found that sole reliance on bibliometric indicators would have led us to miss papers containing important results as judged by expert review. In particular, some papers that were highly rated by experts were not highly cited during the first three years after publication. Tools that link expert peer reviews of research paper quality and importance to more quantitative indicators, such as citation analysis would be valuable additions to the field of research assessment and evaluation.

PMID:
19536339
PMCID:
PMC2695409
DOI:
10.1371/journal.pone.0005910
[Indexed for MEDLINE]
Free PMC Article
PubMed Commons home

PubMed Commons

0 comments
How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for Public Library of Science Icon for PubMed Central
    Loading ...
    Support Center