NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Chung M, Balk EM, Ip S, et al. Reporting of Systematic Reviews of Micronutrients and Health: A Critical Appraisal: Nutrition Research Series, Vol. 3. Rockville (MD): Agency for Healthcare Research and Quality (US); 2009 Mar. (Technical Reviews, No. 17.3.)

Cover of Reporting of Systematic Reviews of Micronutrients and Health: A Critical Appraisal

Reporting of Systematic Reviews of Micronutrients and Health: A Critical Appraisal: Nutrition Research Series, Vol. 3.

Show details


The number of SRs relating micronutrient intake to health outcomes has grown rapidly in recent years. These reviews have been published in a broad range of journals, many with relatively high citation impacts. These trends suggest an increasing acceptance of SRs as a useful way to summarize the data by the nutrition community. SRs of the literature serve as the core of evidence-based guideline development. Dietary guidance issued without pre-specified and transparent evidentiary support may be more prone to errors (173) due to their greater reliance on expert opinion and the potential for omitting important data unknown to the experts. Because of the complex nature of how nutrients are handled and function in the human body, often a large number of linked questions are required for the development of nutrition guidelines. Incorporating currently existing SRs into a new SR can be a cost-effective use of resources but also has potential risks associated with doing so (174). To ensure that future nutrition-related SRs will be of maximal value, the highest standards in their conduct and reporting must be used. Good quality SRs should minimize the likelihood of bias or misinterpretation. SRs are also helpful in identifying knowledge gaps for which specific research agenda or recommendations are needed.

Because of deficiencies in conducting and reporting of SRs in the medical literature, expert panels convened to develop guidelines for SRs. The resulting QUOROM and MOOSE lists have been adopted by SR methodologists and medical journals as standards (13). However, there are several factors that are important for interpreting nutrition research, and thus nutrition SRs, that are not included in the SR quality checklists designed for the medical literature. Thus, we developed a list of 35 items that included the potentially relevant items from QUOROM and MOOSE, along with new nutrition-specific items following the rationale described in Table 1.

Our analysis of a large cohort of nutrition SRs found that 14 of the 35 items commonly were not reported or considered in the SRs; of these, six concerned variables that are unique to the field of nutrition. Moreover, we identified deficiencies in reporting of eight (of 28) items on the clarity or transparency of methods and results (Table 3). While there is currently no consensus on nutrition quality rating issues, the reporting items used in this analysis were selected because of the likelihood that they would have generic utility across SRs conducted for different purposes. It is, however, also recognized that exceptions to generic reporting standards for nutrition SRs may be needed for specific SR applications (e.g., regulatory applications). In these cases, justification for the exceptions could be noted in the design and reporting of the SR. This standardization and transparency would clarify the applicability of a SR for purposes other than those for which it was designed and enhance comparisons of results across SRs on similar topics.

Some generic quality issues are applicable to all SRs. For example, a comprehensive and transparent search strategy, with adequate justifications for inclusion or exclusion of specific studies, is needed to ensure an unbiased selection of studies for SRs and to improve understanding of how the SR was conducted. Furthermore, searching for unpublished data and comparing them to published data could shed some insights on the potential impact of publication bias (175). There is an underlying suspicion of publication bias against studies having either null or negative outcomes (176). It is important to note that there are no reliable methods to measure publication bias. Studies have shown that the most frequently used method to assess publication bias (funnel plots) can be misleading (177–179). Quality assessment of the primary studies is essential for the evaluation of validity and the overall strength of the conclusions in a SR.

The strength of SRs and meta-analyses relies not only on the validity of the included primary studies, but also on the clarity of the reporting of the SR itself. Although good reporting does not necessarily equate valid results, good reporting provides useful information for evaluating the validity of the findings. Our analyses showed that more SRs of interventional studies than those of observational studies (54 percent versus 3 percent, respectively) used quality scales or checklists to assess the methodological quality of the primary studies included. Without quality assessments, the validity of the included primary studies is unclear and the impact of the potential biases in the primary studies on the conclusions of a SR cannot be assessed. Furthermore, SRs of interventional studies met more quality criteria than SRs of observational studies. This finding could be explained in part by the lack of reporting standards for observational studies (this is in contrast to RCTs, many of which have adopted the CONSORT reporting standards (180;181)). Recently, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist (182) was developed to improve the reporting quality of observational studies. It is important to note that CONSORT and STROBE are aimed at guiding authors to report the findings of the primary studies; they were not designed as tools for assessing the quality of the primary studies included in SRs or meta-analyses. Our analyses also showed that the proportion of reporting criteria met was significantly, positively correlated with both the journals’ impact factors and yearly citation numbers. This suggests that SRs of higher reporting quality are more likely to be published on higher impact journals and had wider research dissemination.

In summary, our findings suggest that the reporting quality of SRs has improved since publication of the reporting standards but the reporting of nutrition or diet variables has not. This limits their potential value to help in formulating nutrition-related guidelines, recommendations or research agendas. Reporting standards of SRs should be tailored for specific types of research to help the users of these SRs interpret the results. An improvement in the reporting quality of meta-analyses of RCTs in the critical care literature was documented after the publication of QUOROM (183). Our analysis documents the lack of consistent standards in conducting and reporting SRs of nutrition-related topics. It also provides useful insights on key reporting items for nutrition SRs. In addition to study design features that are important in reducing bias in all studies, for nutrition-related interventional studies it is critical to report the source and dose of the intervention, such as brand names or components (or formulation) of the nutrient supplements, or foods (or recipe) in the nutrition interventions, and the amount of nutrients (or the doses) in the interventions and intervention regimens (e.g., the number of times per day). It is also important to report the baseline nutrient exposures or the background diet (i.e., baseline dietary intake levels or the levels of the biomarker of intakes) in the study population, as the background diet could be one source of heterogeneity (i.e., differential effects of nutrient supplementations on health outcomes) in a SR or meta-analysis. For the nutrition epidemiological studies, it is important to report the methods or instruments for assessing intakes of nutrient exposures, ranges or distributions of the nutrient exposures, measurement errors of the diet or nutrient variables, and the potential impact of the errors from assessing the nutrient exposures on the nutrient-outcome association.

Improving the methodological and reporting quality of nutrition SRs ought to produce more accurate, less biased summaries of the evidence and will allow users of the SRs – general readers, guideline developers, policy makers, and others – to have a better understanding of what evidence the SRs summarize and what biases may exist. While there is room for revision of the quality items for nutrition SRs based on expert consensus, better adherence to the quality items analyzed here is likely to improve the usefulness and acceptance of nutrition SRs.


Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...