NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Chung M, Balk EM, Ip S, et al. Reporting of Systematic Reviews of Micronutrients and Health: A Critical Appraisal: Nutrition Research Series, Vol. 3. Rockville (MD): Agency for Healthcare Research and Quality (US); 2009 Mar. (Technical Reviews, No. 17.3.)

Cover of Reporting of Systematic Reviews of Micronutrients and Health: A Critical Appraisal

Reporting of Systematic Reviews of Micronutrients and Health: A Critical Appraisal: Nutrition Research Series, Vol. 3.

Show details

3Results

The MEDLINE® search identified 3,796 citations; of which 259 full-text articles were retrieved and examined to confirm their eligibility. Three additional articles were identified from citations in retrieved SRs. A total of 141 SRs (105 with and 36 without meta-analyses) were eligible (15;22161). Among these, 90 included interventional studies alone, 31 included observational studies alone, and 20 included both types of study designs (Figure 1). Among the reviews that did not meet eligibility criteria, nine publications stated they were a SR and/or meta-analysis, or evidence-based review but that did not meet the criteria of our predetermined definition, mostly because the authors did not state the eligibility criteria for primary studies reviewed (162–170). Among the eligible reviews, alternative names used for SR included evidence-based review, evidence review, critical review, qualitative overview, overview, in-depth review of the evidence, and review.

Figure 1. Selection process and the number of the included and excluded systematic reviews.

Figure 1

Selection process and the number of the included and excluded systematic reviews.

The earliest SR identified was published in 1989 (51). Half of the SRs were published since 2003. There has been a steady increase in the number of SRs published annually; the number of published SRs tripled from 1999 to 2006 (Figure 2). The number of primary studies included in each SR ranged from 1 to 264; 60 percent of the SRs included fewer than 20 primary studies. A wide variety of potential relationships between micronutrients and health outcomes were examined (Table 2). Of 141 SRs, 88 (62 percent) evaluated clinical outcomes, 35 (25 percent) intermediate outcomes, and 18 (13 percent) both types of outcomes. CVD and cancers were the most common outcomes reported.

Figure 2. Annual publication of systematic reviews of micronutrients and health (search ended Week 2 July 2007).

Figure 2

Annual publication of systematic reviews of micronutrients and health (search ended Week 2 July 2007).

Table 2. Topics covered in the 141 qualifying systematic reviews linking micronutrients and health outcomes.

Table 2

Topics covered in the 141 qualifying systematic reviews linking micronutrients and health outcomes.

Reporting characteristics of the 141 SRs linking micronutrients and health outcomes are summarized in Table 3. Items that SRs commonly did not report or include were: whether literature searches in multiple languages (30 percent of SRs), whether unpublished data were included (28 percent), descriptions of the nutrition status of the population at baseline (32 percent), use of quality scales or items to assess validity (29 percent), dose-response relationships of the nutrient-outcome association (35 percent), assessments or discussions of publication bias (40 percent), use of a flow diagram for the number of studies included and excluded (26 percent), evaluations of potential confounding or interactions of the nutrient-outcome association (49 percent), specific future research recommendations (35 percent), sources of the nutrient interventions (i.e. brand names, components or formulation of the nutrient supplements, or foods or recipes) (46 percent), baseline nutrient exposures in the study population (28 percent), ranges of the nutrient exposures (33 percent), errors from assessing nutrient exposures (i.e. errors from dietary assessments or biomarker assays) (31 percent), and potential impacts of the errors from assessing the nutrient exposures on the findings (24 percent). The definitions of adequate reporting of the 35 reporting items are described in Table 1.

Table 3. Reporting characteristics in systematic reviews (with or without meta-analyses) of micronutrients and health outcomes.

Table 3

Reporting characteristics in systematic reviews (with or without meta-analyses) of micronutrients and health outcomes.

Factors Associated With the Reporting Quality

On average, SRs that linked micronutrients and health outcomes met 57 percent (IQR: 48 percent, 66 percent) of our reporting criteria. SRs that included only interventional studies met a higher proportion of reporting criteria (median: 62 percent, IQR: 51 percent, 72 percent) than those with only observational studies (median: 53 percent, IQR: 47 percent, 60 percent) or both study designs (median: 47 percent, IQR: 39 percent, 52 percent) (P<.001). (Figure 3) There were statistically significantly more SRs of interventional than observational studies that reported a search for unpublished studies (40 percent versus 3 percent), described the reasons for study exclusions (64 percent versus 42 percent), used quality scales or items to assess validity (39 percent versus 3 percent), and included a flow diagram of the number of studies included and excluded (37 percent versus 6 percent). There were significantly fewer SRs of interventional than observational studies that analyzed the potential confounding or interactions of the nutrient-outcome associations (37 percent versus 71 percent) and that made specific future research recommendations (29 percent versus 52 percent).

Figure 3. Proportion of reporting criteria met among 141 systematic reviews of micronutrients and health.

Figure 3

Proportion of reporting criteria met among 141 systematic reviews of micronutrients and health.

We examined the association between the reporting quality and publication of the MOOSE and QUOROM reporting standards for SRs by testing the difference in reporting quality comparing those published before publication of these standards and SRs published 3 years after. There were 115 SRs that were published before 1999 (n=31) or since 2003 (n=84); articles published between 1999 and 2002 were excluded for being conducted too close in time to the publication of the reporting standards. Before the reporting standards, SRs met a lower proportion of our reporting criteria than after (median: 50 percent versus 59 percent, P=0.01), suggesting that the overall reporting quality of SRs linking micronutrients and health outcomes has improved since publication of the reporting standards. In contrast, the reporting of nutrition variables remained unchanged (median: 33 percent versus 38 percent, P=0.7) (Figure 4).

Figure 4. Proportion of reporting criteria met comparing systematic reviews published before 1999 to 3-year after publication of QOUROM and MOOSE.

Figure 4

Proportion of reporting criteria met comparing systematic reviews published before 1999 to 3-year after publication of QOUROM and MOOSE.

Of the 141 SRs, 128 were published in 84 journals with impact factors that ranged from 0.3 to 25.8; 13 SRs (8 with meta-analyses) were published in journals not indexed in the Journal Citation Reports®, therefore, they were excluded from the relevant analyses. There was a positive correlation between the proportion of reporting criteria met and the journals’ impact factors (r=0.35, P<0.001), indicating that SRs published in higher impact journals were more likely to have met a high proportion of our reporting criteria. The median yearly number of citations attributable to the SRs was 4, ranging from 0 to 100 (excluding an outlier (109) that has had 2,128 citations since 1995). The proportion of reporting criteria met was not significantly correlated with the yearly number of citation (r=0.11, P=0.18) but both correlation coefficient and statistical significance improved after excluding the outlier SR (r=0.26, P=0.003).

SRs containing meta-analyses (n=105) met a higher proportion of our reporting criteria compared to the 31 SRs without meta-analyses (median: 62 percent versus 48 percent, P<0.001). SRs containing meta-analyses were also published in journals with higher impact factors (median 4.3 versus 2.8, P=0.001) and received more yearly citations (median: 16 versus 6, P=0.001).

Quality Assessment of the Primary Studies

There were 58 SRs (49 of interventional studies, 1 of observational studies, and 8 of both designs) that used quality scales or checklists to assess the methodological quality of the primary studies. The most commonly used were Jadad (171) and Schulz (172) quality scores or checklists, which were designed to assess the adequacy in the conduct of RCTs. The one SR of observational studies used a modified quality checklist, which was originally developed to evaluate the quality of interventional studies (an unpublished thesis). Among the eight SRs of both intervention and observational studies, eight different quality scales or checklists were used. Seven of the eight SRs used single quality scales (e.g., good, fair, or poor) for both intervention and observational studies. The definitions (or the quality items considered) of these quality scales varied. One SR used separate quality checklists for intervention (Jadad) and observational studies.

Views

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...