Send to:

Choose Destination
See comment in PubMed Commons below
JAMA. 2006 Apr 19;295(15):1801-8.

Second-order peer review of the medical literature for clinical practitioners.

Author information

  • 1Health Information Research Unit, Department of Clinical Epidemiology and Biostatistics, Michael G. DeGroote School of Medicine, McMaster University, Faculty of Health Sciences, Hamilton, Ontario, Canada.



Most articles in clinical journals are not appropriate for direct application by individual clinicians.


To create a second order of clinical peer review for journal articles to determine which articles are most relevant for specific clinical disciplines.


A 2-stage prospective observational study in which research staff reviewed all issues of over 110 (number has varied slightly as new journals were added or discarded from review but number has always been over 110) clinical journals and selected each article that met critical appraisal criteria from January 2003 through the present. Practicing physicians were recruited from around the world, excluding Northern Ontario, to the McMaster Online Rating of Evidence (MORE) system and registered as raters according to their clinical disciplines. An automated system assigned each qualifying article to raters for each pertinent clinical discipline, and recorded their online assessments of the articles on 7-point scales (highest score, 7) of relevance and newsworthiness (defined as useful new information for physicians). Rated articles fed an online alerting service, the McMaster Premium Literature Service (PLUS). Physicians from Northern Ontario were invited to register with PLUS and then receive e-mail alerts about articles according to MORE system peer ratings for their own discipline. Online access by PLUS users of PLUS alerts, raters' comments, article abstracts, and full-text journal articles was automatically recorded.


Clinical rater recruitment and performance. Relevance and newsworthiness of journal articles to clinical practice in the discipline of the rating physician.


Through October 2005, MORE had 2139 clinical raters, and PLUS had 5892 articles with 45 462 relevance ratings and 44 724 newsworthiness ratings collected since 2003. On average, clinicians rated systematic review articles higher for relevance to practice than articles with original evidence and lower for useful new information. Primary care physicians rated articles lower than did specialists (P<.05). Of the 98 physicians who registered for PLUS, 88 (90%) used it on 3136 occasions during an 18-month test period.


This demonstration project shows the feasibility and use of a post-publication clinical peer review system that differentiates published journal articles according to the interests of a broad range of clinical disciplines.

[PubMed - indexed for MEDLINE]
PubMed Commons home

PubMed Commons

How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for Silverchair Information Systems
    Loading ...
    Write to the Help Desk