Meta-Analysing Methodological Quality of Published Research: Importance and Effectiveness

The Inappropriate design of experimental studies in medicine inevitably leads to inaccurate results and biased conclusions. The aim of our study was to compare prevalence of implementing basic principles of experimental design in preclinical experimental studies published in international journals with low and high impact factor. The samples for analysis ware randomly chosen among publications retrieved from PubMed and Web of Science (WoS). Implementation rate of basic experimental research principles (local control, randomization and replication) was established by careful reading of the sampled publications and their checking against predefined criteria. The difference in number of satisfied criteria among the groups was not significant, however, number of citations was significantly higher in the group of studies published in high-impact factor journals (30.5 ± 38.5 vs 2.6 ± 4.1, p=0.000). The studies published in low-IF journals less frequently used pseudo replication (30% vs 56%, p=0.000) and more often randomized their units of observation (40% vs 5%, p=0.000). Prevalence of experimental preclinical studies that did not implement completely basic principles of research design was high in both low- and high-impact factor journals. Although much more cited, studies published in high-impact factor journals bore the same risk of incorrectness, bias, and consequent misleading of future researchers.