NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Ammerman A, Lindquist C, Hersey J, et al. The Efficacy of Interventions to Modify Dietary Behavior Related to Cancer Risk. Rockville (MD): Agency for Healthcare Research and Quality (US); 2001 Jun. (Evidence Reports/Technology Assessments, No. 25.)

  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Cover of The Efficacy of Interventions to Modify Dietary Behavior Related to Cancer Risk

The Efficacy of Interventions to Modify Dietary Behavior Related to Cancer Risk.

Show details

2Methodology

In this chapter, we outline our strategy for identifying and screening articles relevant to determining the impact of behavioral interventions on dietary change related to cancer risk. We describe the process of abstracting relevant information from the eligible articles and generating the summary Evidence Tables, which report key details about the study methodology and findings of the articles we reviewed. Finally, we detail the secondary analysis strategy we employed to explore the efficacy of interventions in changing dietary outcomes.

Literature Identification

Literature Search

The first step in the literature identification process involved identifying key search terms and relevant databases on which to perform the literature search. The Literature Review Specialist, in conjunction with the Study Director and the Scientific Director, determined the search terms used in the analysis. During this process, several literature searches revealed the disparate nature of the literature and the necessity of using complex, interactive strategies to narrow the list of potential articles for abstraction. For example, because "intervention" is not included as a MeSH heading, terms such as food habits, health behavior, and health promotion, which are less precise, had to be used. In addition, our search strategy included cardiovascular diseases and other chronic diseases (e.g., diabetes) with similar dietary intervention outcomes. Despite numerous attempts to limit the records to the literature pertaining to dietary interventions rather than general nutritional epidemiology studies, each search produced significant "noise" from the epidemiologic literature. Because many of the abstracts found by a standard literature search did not include adequate information for determining inclusion/exclusion criteria, more than 300 full articles were pulled for review by our Scientific Director and our Study Director in order to select articles that met our criteria.

Searches were performed in the following six databases: (1) MEDLINE , the U.S. National Library of Medicine (NLM) database; (2) EMBASE; (3) PsycINFO; (4) the Cumulative Index to Nursing and Allied Health Literature (CINAHL); (5) AGELINE (produced by the American Association of Retired Persons [AARP]); and (6) AGRICOLA (AGRICultural OnLine Access). This list of databases included PsycINFO as specified by both AHCPR and NCI to ensure that we captured the fullest range of literature on behavioral change.

"Exploding" the search term Diet and/or Nutrition (i,e., including the more-specific terms branching off a primary search term) yielded more than 100,000 results using MEDLINE, EMBASE, and AGRICOLA. However, fewer (about 2,000) articles were identified in the other databases. Because of the unwieldy nature of the initial search, the primary search terms included were (1) health behavior; (2) attitude to health; (3) health promotion; (4) behavior change; (5) food habits; (6) fruit; (7) vegetables; (8) prevent; (9) counsel; (10) cardiovascular disease; (11) cancer; and (12) neoplasms. Other searches specifically focused on diabetes studies (type 2 diabetes) as suggested by our Technical Expert Advisory Group (TEAG).

After applying the exclusionary terms to the combined searches (as shown in detail in Tables 1 through 12), we identified 1,106 articles, of which 877 were unduplicated records. Additional articles were added from reviews of reference lists and from recommendations from members of our TEAG and from peer reviewers of the draft evidence report. We added 30 records based on suggestions from the TEAG and peer reviewers of the draft evidence report. This yielded 907 records that potentially fulfilled our review criteria. We screened these articles using the inclusion criteria described below. Our formal cut-off date for article inclusion was publication before August 1999. However, we added selected articles published after that date when the TEAG felt that inclusion of the articles was essential to the report.

Table 1. Expanded literature search results using MEDLINE (1966 to present).

Table

Table 1. Expanded literature search results using MEDLINE (1966 to present).

Table 2. Literature search results using EMBASE (Excerpta Medica).

Table

Table 2. Literature search results using EMBASE (Excerpta Medica).

Table 3. Literature search results using PsycINFO.

Table

Table 3. Literature search results using PsycINFO.

Table 4. Literature search results using Cumulative Index to Nursing and Allied Health Literature (CINAHL).

Table

Table 4. Literature search results using Cumulative Index to Nursing and Allied Health Literature (CINAHL).

Table 5. Literature search results using AGELINE.

Table

Table 5. Literature search results using AGELINE.

Table 6. Literature search results using AGRICOLA.

Table

Table 6. Literature search results using AGRICOLA.

Table 7. Literature search results pertaining to "Meta-Analysis and Diet Counseling" using MEDLINE.

Table

Table 7. Literature search results pertaining to "Meta-Analysis and Diet Counseling" using MEDLINE.

Table 8. Literature search results pertaining to "Counseling" using MEDLINE.

Table

Table 8. Literature search results pertaining to "Counseling" using MEDLINE.

Table 9. Literature search results pertaining to "Meta-Analysis and Cancer" using MEDLINE.

Table

Table 9. Literature search results pertaining to "Meta-Analysis and Cancer" using MEDLINE.

Table 10. Literature search results pertaining to "Diet and Cancer" using MEDLINE.

Table

Table 10. Literature search results pertaining to "Diet and Cancer" using MEDLINE.

Table 11. Literature search results pertaining to "Diet Therapy" using MEDLINE.

Table

Table 11. Literature search results pertaining to "Diet Therapy" using MEDLINE.

Table 12. Literature search summary.

Table

Table 12. Literature search summary.

Inclusion/Exclusion Criteria

In developing the inclusion and exclusion criteria for the literature, we created a series of parameters that progressively narrowed the population of articles to be abstracted. Our inclusion/exclusion criteria are presented in Table 13. The first stage of our screening involved excluding articles that did not report original research (such as review articles) or studies that did not report the results of dietary interventions. We also excluded articles that were published before 1975 (regardless of when the intervention actually took place); conducted outside of North America, Europe, or Australia; and published in languages other than English.

Table 13. Inclusion/exclusion criteria.

Table

Table 13. Inclusion/exclusion criteria.

Next, we excluded studies based on population characteristics. The general population on which we focused in this report was human adults, adolescents, and children; therefore, studies on infant populations were excluded. We decided to include both healthy populations and populations at high risk of disease as long as subjects were not restricted to a hospital or chronic care setting. However, populations with type 1 diabetes (insulin-dependent) relying on regimented diets (such as patients with renal disease) were excluded. Type 2 diabetes patients taking insulin were included. Finally, we excluded institutionalized populations (such as prisoners and nursing home residents).

Regarding study design, we included both randomized controlled trials (RCTs) and nonrandomized controlled trials (non-RCTs) that had nonequivalent control or comparison group designs. Sample size was used in our screening; studies with fewer than 40 subjects at follow-up were excluded.

Several characteristics pertaining to the intervention were considered as criteria for inclusion. Interventions of all types (e.g., individual dietary counseling, group nutrition classes, social support groups) and settings (e.g., school, workplace, media, health care setting, policy changes) were included in this review, and no minimum level of intervention intensity or duration was required for inclusion. However, the intervention must have allowed dietary intake to be freely chosen by the participant. Thus, studies that provided prepackaged meals to subjects were excluded.

Because dietary outcomes considered in this report were based on relevance to risk of cancer, we included only studies reporting results for fruit and vegetable intake or dietary fat intake. However, in our abstraction of selected articles, we recorded limited information for fiber intake (this information is presented in the Evidence Tables, which are described in a later section of this chapter). The decision to focus on intake of dietary fat and fruits and vegetables was based on discussions with our TEAG, during which it was generally agreed that the evidence for the relationship between fiber and cancer risk was inconclusive. Other criteria that influenced our decision were the prevalence of dietary outcomes reported in the literature and the extent to which dietary fat and fruits and vegetables are emphasized as dietary goals in behavioral interventions.

Follow-up (i.e., post-intervention) results for these outcomes must have been reported in order for the article to be included. Although results for biochemical indicators (e.g., total cholesterol, low-density lipoprotein cholesterol, and carotenoids) and behavioral mediators (e.g., dietary knowledge, stages of change, dietary self-efficacy) were abstracted in our review of the articles, these outcomes were not necessary for an article to be included.

We did not use dietary measurement methodology as an exclusion criterion, so we identified articles with a wide variety of assessment techniques. All dietary intake data were self-reported in the articles we reviewed, using methodologies such as dietary recalls, food records or diaries, dietary histories, or food frequency questionnaires. Because such techniques are based on self-reported information, they are subject to bias from participant memory and judgment, which can result in underreporting of energy intake (which is particularly problematic among overweight individuals). Indeed, social desirability bias is especially likely to be a source of error in behavioral intervention studies, because participants are taught about the "right" foods to consume and may be less likely to report intake of "unhealthful" foods. Thus, the limitations associated with self-reported dietary intake should be kept in mind when interpreting the results presented in this report.

Using the exclusion criteria described in this section, we reviewed the 907 articles identified in our literature search (either by abstract or by full article review) and considered them for potential inclusion. Of these articles, we excluded 803 records (see Table 14 for reasons for exclusion), resulting in a total of 104 articles that were included in our review (referenced in the Evidence Tables in the order they appear in the text).19-122

Table 14. Reasons for Exclusion.

Table

Table 14. Reasons for Exclusion.

Data Collection

The data collection process involved abstracting relevant information from the eligible articles and generating summary Evidence Tables that present the key details and findings for the articles. A team of two trained abstractors independently completed a detailed Data Abstraction Form -- which for each eligible article elicited relevant information about the study methodology and results -- and a Quality Rating Form, which rated the quality of the article. The Study Director used the forms and the original articles to generate summary Evidence Tables. Quality control functions were performed by the Scientific Director and a senior abstractor, who re-reviewed the information reported in the Evidence Tables and reconciled discrepancies between the abstractors.

Abstractors and Training

The RTI-UNC EPC used abstractors with two types of backgrounds for the data extraction process: content or clinical expertise and strong methodological skills. The clinical abstractors were Semra Aytur, MPH; Kerry-Ann daCosta, PhD; Denise D. Dickinson, MPH; Alyssa Ghiradelli, MPH, RD; Christine S. Hardy, RD; and Hugh C. Law, RD. All had prior research experience. The methods abstractors were Tracy L. Bouchard-Cyr, MSPH; Nancy A. Davis, MSHE, MPH; Ho-Jui Tung, MPH; Kimberly Truesdale, MSPH; and Carole Toselli, MD.

All abstractors attended two formal training sessions. At the first session we explained the process and goals of the abstraction. Following the training, the abstractors were sent home with an article to review. We then reconvened the group and, through a review of the test article, ensured that the reviewers understood what was expected from their work. For example, we instructed abstractors that when inconsistencies arose between results stated in the text of an article and those presented in tabular form, they were to take data from the text. They were also told to extract precisely what was contained in the article and to reserve any opinions about the contents to notes in the margins. At the completion of the training, the data abstraction process began. The Scientific Director and the UNC Research Coordinator monitored progress. Any problems or questions encountered by the abstractors were routed though the Research Coordinator to the appropriate senior staff member.

Change in Abstraction Process

Initially we had both a clinical and a methodological abstraction independently completed for each eligible article. The abstractors met to resolve differences, and a reconciled form, with individually completed forms attached, was submitted to the Research Coordinator. However, despite extensive abstractor training, the complexity and high variability in the reporting of behavioral interventions required a comprehensive re-review of all data sources by the Study Director or the Scientific Director. To make the process more time- and cost-efficient while also producing the most accurate reporting of the data, we modified our procedure to have a single trained abstractor complete the abstract form, followed by a detailed review of all information by a senior member of the project team. We found, upon review by the Study Director and the Scientific Director, that this process generated a higher level of accuracy and detail in the summary database while maintaining review of the abstraction by two people.

Data Abstraction Form

The Data Abstraction Form (included in Appendix D) was used to extract the relevant information presented in the article and to confirm that the study fulfilled the inclusion criteria described previously. The Study Director and the Scientific Director worked with core staff and the TEAG to develop the Data Abstraction Form. This form was developed with extensive communication between methodologists and researchers. We began by identifying salient study characteristics and dietary outcomes; we then created specific items eliciting information for the following constructs: intervention characteristics, population characteristics, study design, and the statistical results of the study. Specific information abstracted from the articles included the following:

  • The theoretical framework used in the intervention
  • The setting of the intervention
  • Key components of the intervention (e.g., classes, individual counseling, cafeteria modifications)
  • The delivery of the intervention (e.g., physician, registered dietitian)
  • The nutrition message of the intervention
  • Special features of the intervention (e.g., individually tailored components, ethnic specificity)
  • The intervention duration and intensity
  • The demographic composition of the sample (including gender, age, race, income, health status)
  • Study design
  • Duration of follow-up
  • Participation and retention rates
  • Measurement (including information on the validity and reliability of the dietary assessment technique) and statistical results for fruit and vegetable intake
  • Measurement (including information on the validity and reliability of the dietary assessment technique) and statistical results for dietary fat intake
  • Statistical results for other dietary outcomes (including fiber- and calcium-related outcomes)
  • Measurement and statistical results for biochemical outcomes related to fruit and vegetable or dietary fat intake (including plasma carotenoids and blood lipids)
  • Statistical results for behavioral mediators

The Data Abstraction Form was developed after extensive pretesting by the staff and the abstraction team. To improve the quality of the abstraction process, a comprehensive guide accompanied the final form. After several weeks of using the final version of the form, we decided to have abstractors simply record the table and page numbers for the statistical results on the Data Abstraction Form rather than attempt to transcribe the complete statistical information from the article to the form (with the statistical results entered directly into the Evidence Tables, with re-review by the Scientific Director and a senior abstractor).

Rating the Quality of the Evidence

During the data abstraction process, the abstractors completed a Quality Rating Form for all eligible articles. The scoring of this form for each article was reviewed by the Study Director, the Scientific Director, or the senior abstractor.

Although no consensus exists on criteria for determining the quality of behavioral intervention research, this issue merits considerable attention because it is likely to influence the degree of credibility surrounding study results. Judging the quality of articles in systematic evidence reports is necessarily multidimensional, including features associated with the quality of the study design and intervention as well as the quality of the write-up itself.

Our approach (as shown in Appendix E) was to identify and rate essential features of both the study description and the methodology, weighing the latter component more heavily. Some degree of subjectivity in our approach to rating the quality of the evidence was unavoidable. Some of our criteria did involve the judgment of the abstractor (and reviewer), and even relatively objective information (e.g., sample size) was not always consistently reported in the articles we reviewed. However, the fact that all quality scores were reviewed by a senior member of the project staff resulted in the consistent application of our scoring procedure.

The quality score assigned to each article was a numerical value ranging from 0 to 100. This score was based on the following factors:

  • Whether the intervention was theoretically based (5 points)
  • Whether the research design involved random allocation of individuals/units to treatment groups (10 points)
  • The sample size (10 points)
  • The duration of follow-up (10 points)
  • The retention rates (10 points)
  • The description and validity of the dietary assessment tool (5 points)
  • Whether changes in biochemical outcomes were explored (5 points)
  • Whether analysts were blind to the assignment of treatment groups (5 points)
  • The generalizability of the results (based on the representativeness of the sample and the practicality of the intervention) (10 points)
  • The quality of the description of the intervention (including relevant details about the setting, components, delivery, duration, and intensity of the intervention) (10 points)
  • The quality of the description of the study population, recruitment strategy, and inclusion/exclusion criteria (10 points)
  • The quality of the description of the variable measurement and statistical analysis procedure (10 points)

Methodological features of considerable importance to intervention research in general include study design (with higher quality scores associated with randomized controlled trials), sample size, duration of follow-up, retention rates, whether analysts were blind to treatment assignment, and the generalizability of results. Additional methodological features salient to dietary outcomes include the validity of the dietary assessment tool and the use of biochemical indicators to validate changes in dietary behavior. A unique criterion particularly relevant to behavioral interventions is whether the intervention was based on or guided by a theoretical framework. We incorporated each of these methodological features into our Quality Rating Form; such criteria were worth 70 percent of the total quality score.

In addition to judging the quality of the study methodology, we considered the quality of the written article (in terms of detail and clarity). The quality of the intervention description was particularly important in accurately identifying the type of intervention. Features such as intervention setting (e.g., school, health care setting, media campaign), intervention components (e.g., dietary counseling, support groups, newsletters), delivery mode (e.g., dietitian, physician, regular classroom teacher), and intervention intensity (number of exposures/contacts) were included in our Quality Rating Form. We also rated the article's description of the study population, recruitment strategy, and inclusion/exclusion criteria.

The final component we incorporated in the Quality Rating Form was the article's description of the variable measurement and statistical analysis procedure. Although articles were not excluded on the basis of the dietary assessment technique employed, the description of the methodology and the validity of the assessment tool were used as a component of the quality score assigned to the article; the validity of the measurement tool had to be specified or referenced in order for the article to receive the maximum number of points for that item. Together, the study description features constituted 30 percent of the total quality score calculated for each article. The complete Quality Rating Form is included in Appendix E.

Development of the Evidence Tables

Using the Data Abstraction Form, the Quality Rating Form, and the original article, the Study Director generated summary Evidence Tables, which present a concise summary of key intervention characteristics, methodological details, and statistical results for the 104 articles we reviewed. As mentioned previously, the content of these tables was re-reviewed against the original article by either the Scientific Director or the senior abstractor. In the Evidence Tables, we combined multiple articles reporting results for the same study. Thus, the 104 articles we reviewed represented only 92 independent studies, and each of these 92 studies forms a separate entry in the Evidence Tables.

We decided on the content of the Evidence Tables at an early stage in the project so that we could have all relevant information recorded for each article during the abstraction process. We determined relevant study details through extensive discussions with the RTI-UNC team and the TEAG. The format and general organization of the Evidence Tables were guided by previous RTI-UNC EPC evidence reports.

The Evidence Tables presented in this report are separated by intervention setting, resulting in four tables: school-based interventions, health care interventions (which includes studies in which the intervention was conducted in a health care setting and studies for which subjects were recruited from health care settings), worksite interventions, and community/other interventions (which include interventions conducted in homes, churches, and communities). The content of the Evidence Tables, with a brief description and glossary, is presented in Chapter 7.

The Evidence Tables report information on the intervention setting, subject characteristics (including gender, age, race, and risk status), study design, sample size and retention rates, intervention characteristics (including intervention components, delivery, special features, and nutritional message), duration/intensity of the intervention, and duration of follow-up. The measurement approach (including variables, instrument, and statistical analysis strategy) and statistical results are presented for fruits and vegetables, dietary fats, and biochemical indicators. For other dietary outcomes and behavioral mediators, we provided only a brief description of the significance of the intervention effect. The final piece of information included in the Evidence Tables is the quality score assigned to the article.

Analysis Approach

In developing a strategy for synthesizing the statistical results presented in the Evidence Tables, it became evident that two major issues would have to be resolved. The first is that the articles in the Evidence Tables reported multiple outcome measures. The second is the variety in the statistical analysis techniques used to determine the significance of various interventions as well as the actual statistics reported in the articles.

Based on input from our TEAG about the dietary outcomes reported most commonly in the nutrition literature, the relevance of dietary outcomes to cancer risk, and current dietary recommendations (which are typically the targeted information in behavioral interventions), we included the following outcomes: for fruits and vegetables, total daily servings of fruits and vegetables, daily servings of fruit, and daily servings of vegetables; for dietary fat, total fat as a percentage of energy intake, saturated fat as a percentage of energy intake, and total fat in grams. However, in the interest of including more studies with a variety of designs, we also analyzed other outcomes related to fruits and vegetables and dietary fats. The definition of fruits and vegetables as outcomes is a topic of considerable debate. The issues include whether high-fat, starchy vegetables such as french fries are counted as vegetables and whether fruit juices are considered servings of fruit. Appropriate serving sizes are similarly a topic of contention. We had to rely on the definitions used in the articles we reviewed (because we could not separate out certain foods and create our own definitions), and this factor led in turn to a lack of standardization in our secondary analyses. Fat outcomes also varied. For example, fat-related behaviors (such as substituting skim milk for whole milk, trimming the fat from meat) and intake of individual foods or food groups (such as high-fat meat, butter, or cream) were commonly reported as outcomes in the studies we reviewed. It was a challenge to identify key outcome variables on which to focus, while still including less commonly reported outcomes in the analyses. As described in further detail below, we developed a three-tiered approach for our secondary analysis of dietary outcomes that used different strategies for "key" outcomes and "nonkey" outcomes.

The second issue (diversity in statistical reporting) also had a major impact on our analysis strategy. In the articles we reviewed, investigators took several approaches to determine the statistical significance of the intervention effect. Common approaches included the following: analyzing the interaction between time and treatment group membership; comparing differences in means between the intervention and control group(s) at follow-up (either with or without controlling for baseline values); and using a repeated measures approach to determine the change in the intervention and control group(s) from baseline to follow-up (with results typically reported separately for the intervention and control group). Concomitant with the diversity in analysis approaches (with multiple approaches often reported for a single article) was nonuniformity in the statistics reported. Mean values were commonly reported among the studies we reviewed, but the reporting of the variance was extremely inconsistent; studies reported standard deviations, standard errors, 95 percent confidence intervals, ranges, or, quite commonly, no indicator of variance at all. Similarly, the significance of effects was inconsistently reported, with actual p values or p value "cut-offs" (e.g., <0.05, <0.01) often unreported, particularly for nonsignificant findings. Thus, because of unique journal requirements and editorial policies (particularly limiting the amount of space available to authors), the statistics necessary to conduct appropriate secondary analyses are often inconsistently reported.

Five additional issues inherent to the design of the studies influenced the level of consistency in statistical reporting: (1) using a cross-over design, (2) including multiple intervention groups in the study, (3) reporting results from multiple follow-up periods, (4) reporting results separately for population subgroups (e.g., males and females), and (5) applying inconsistent dietary measurement techniques. These design and reporting features, while appropriate and desirable for understanding the impact of individual interventions, are difficult to incorporate into secondary analyses and complicated our efforts to synthesize the results of the interventions reviewed.

A final issue, which is an inherent limitation of reviews of published literature and secondary analyses conducted on such literature, is the potential for publication bias. Our review must necessarily rely on previously published, peer-reviewed work, but the pool of articles we reviewed is highly likely to have been biased toward "positive" findings, or findings more likely to support the relationship between behavioral interventions and dietary change (with negative findings being less likely to be submitted or accepted for publication). As a result, our secondary analyses will reflect this bias -- a likelihood that must be considered when interpreting our results.

To accommodate both the statistical diversity and the variety of outcomes reported in the articles we reviewed, we developed a three-tiered secondary analysis strategy that incorporated different analytic techniques for particular outcomes and for specific types of statistical information reported in the articles. As mentioned previously, although we included articles employing consistent units of measurement (such as percentage energy), we could not ensure consistency in dietary assessment techniques because of the wide variety of measurement strategies and often unclear methodological descriptions.

The three tiers of the analysis strategy (described in further detail in the subsequent sections) are the following:

  1. Meta-analysis (based on groupings of articles reporting results for comparable populations) of the change (and variance) between intervention and control groups over time for two outcomes:
    • total daily servings of fruits and vegetables
    • daily intake of total fat as a percentage of energy intake
  2. Standardized, quantitative analysis of the change between intervention and control groups from baseline to follow-up for a set of key outcomes:
    • total daily servings of fruits and vegetables
    • total daily servings of fruits
    • total daily servings of vegetables
    • daily intake of total fat as a percentage of total energy intake
    • daily intake of saturated fat as a percentage of total energy intake
    • daily intake of total fat in grams per day
  3. Semiquantitative analysis summarizing whether the reported intervention effect was significant for the following sets of dietary outcomes among all articles:
    • fruits and vegetables
    • fruits
    • vegetables
    • total fat
    • saturated fat
    • general fat intake scores
    • specific high-fat foods or high-fat behaviors
    • specific low-fat foods or low-fat behaviors

The primary goal of each of the three secondary analysis strategies was to determine the overall effectiveness of dietary interventions at changing dietary behavior. Secondary goals included a determination of the relative effectiveness of different types of interventions and among different population subgroups at changing dietary behavior. Strategies for determining the relative effectiveness of interventions are discussed in the final section of this chapter.

Meta-Analysis

The first approach in our determination of the efficacy of dietary interventions was a statistical meta-analysis. Our initial step in conducting the meta-analysis was to determine appropriate outcomes on which to focus. Based on our review of the most commonly reported dietary outcomes in the interventions included in our study, as well as discussions with our TEAG regarding common dietary goals emphasized in dietary interventions and dietary outcomes most consistently linked to cancer risk, we selected daily servings of fruits and vegetables and total fat as a percentage of energy intake as the two most appropriate outcomes for our meta-analysis.

The second step of our meta-analysis was to determine the feasibility and utility of conducting a meta-analysis on the outcomes we selected, based on criteria identified by our meta-analysis expert. Specifically, the following essential information must have been reported in the article:

  • results for both the intervention group(s) and the control group
  • either (a) means at baseline and follow-up or (b) means at baseline and the mean change between baseline and follow-up
  • the standard deviation, the standard error, or the 95 percent confidence interval for the means

As described in Chapter 3, we determined that we had an insufficient number of eligible articles to conduct a meta-analysis of fruit and vegetable intake. We based this conclusion primarily on the small number of articles including daily servings of fruits and vegetables as an outcome (n = 14) and on the inadequacy of the statistical reporting among these articles. In addition, statistically eligible articles were not considered substantively comparable enough to merit producing a combined estimate of the intervention effect; the problems were wide variability in population characteristics and in the type and duration of intervention. We did identify a sufficient number of articles reporting the necessary statistical information for total fat (percentage of energy) intake, and thus we conducted a meta-analysis for this outcome.

Of the 45 articles reporting results for total fat (percentage of energy) as an outcome, 27 articles met the statistical criteria for the meta-analysis. Based on the advice of our TEAG, we contacted authors of "excluded" studies to obtain unpublished information (such as standard deviations) that would enable us to include additional articles in the meta-analysis. We contacted 13 authors (of the excluded articles) and received additional information from two authors, resulting in a final pool of 29 articles eligible for the meta-analysis.

The final step was to organize the eligible articles into meaningful groupings for which results could be combined. Our approach was to identify groupings of articles based on the comparability of population characteristics, interventions received, and duration of follow-up. Based on advice and discussion with our TEAG and our meta-analysis expert, we identified the following groupings:

  • school-based interventions with healthy children
  • worksite and community interventions with healthy adults
  • health care setting interventions with healthy adults
  • health care setting interventions with high-risk adults (including populations at risk of, but not diagnosed with, cancer and type 2 diabetes)
  • health care setting interventions with adults diagnosed with a disease (including cancer, type 2 diabetes, and cardiovascular disease [CVD])

This classification scheme resulted in our assigning 28 of the 29 articles to one of the five groupings. The remaining article (reporting results from a health care intervention on children diagnosed with type 1 diabetes) did not fit any of the groupings and was not included in the meta-analysis.

We then reviewed each study within the five groupings and found a significant amount of heterogeneity. We determined that the considerable differences in study populations, intervention components, intervention intensity, and follow-up times precluded combining all studies within major groupings. However, we did identify studies within each group that we judged were similar enough to combine. We made the appropriate combinations and plotted them together with the results of the other individual studies to facilitate comparison and interpretation.

Across all studies, the contrast of interest was the difference between intervention and control groups in the change over time in percentage of calories obtained from fat (i.e., the difference in differences). A positive mean effect typically indicates that the decrease in consumption of dietary fat in the intervention was greater than the change in consumption of fat in the comparison group. Standard error estimates for each contrast were also required to construct confidence intervals and for combining results for meta-analysis. For some studies, the standard errors were given, while for others they had to be inferred.

Studies that did not explicitly state the standard error of the contrast had (1) a test statistic for the contrast (t or F); (2) the probability of the observed contrast value when the population value is 0, with degrees of freedom; or (3) the standard errors for each group by time mean. When the value of the t statistic was given, the standard error was estimated by C/t, where C was the observed value of the contrast. When an F statistic was given, C/Image f3629_f001.jpg was used. Finally, when a probability was given, it was assumed to be from a two-tailed t-test or an F test with 1 numerator degree-of-freedom. The relevant tstatistic was obtained using the inverse of the cumulative t probability distribution, and that value was applied to the above equation. These two methods yielded proper standard errors that use the within-subjects nature of most of the studies.

When only the standard error estimates for each group by time mean were available, obtaining proper contrast standard error estimates was not possible. Observations were treated as independent, and the standard error was estimated using the pooled variance estimate. This method would result in conservative standard error estimates for some studies because observations within subjects would be non-negatively correlated over time. These studies would be underweighted in subsequent meta-analyses as a result of these exaggerated standard error estimates.

For each study, the theoretical distribution of the contrast estimate was assumed to be normal, with the mean equal to the observed value of the contrast and a variance equal to the squared contrast standard error estimate. When appropriate, information from studies was combined using a fixed effects, variance-weighted meta-analysis. The result was an estimate for the common contrast value and an estimated standard error.

Standardized Analysis of the Magnitude of the Intervention Effect

The second tier of our analysis strategy involved a systematic analysis of the difference in change from baseline to follow-up (among dietary outcomes) between intervention and control groups (i.e., difference-in-deltas). This approach allowed us to use a common metric (difference in the change between groups, expressed as a percentage) to compare the magnitude of changes in dietary intake across interventions. We decided to focus on six "key" outcomes that are commonly reported across the studies in our Evidence Tables and are particularly relevant to cancer risk. These six outcomes were (1) daily servings of fruits and vegetables, (2) daily servings of fruits, (3) daily servings of vegetables, (4) daily intake of total fat (as a percentage of energy intake), (5) daily intake of saturated fat (as a percentage of energy intake), and (6) daily intake of total fat (grams). In addition to being limited to particular outcomes, this strategy was also limited (by necessity) to articles reporting adequate statistical information for the calculation of this metric.

Thus, the first step of our approach was to identify articles eligible for the calculation of the differences in deltas. The requirements for calculation were that the article must have reported:

  • results for both the intervention group(s) and the control group
  • either (a) means at baseline and follow-up or (b) means at baseline and the mean change between baseline and follow-up
As is evident in the requirements above, one advantage of this strategy is that it enabled us to include more articles than had been included in the meta-analysis because it did not require any indication of variance.

The calculation of the difference-in-deltas involved several steps: (1) computing the change in mean intake from baseline to follow-up for the intervention group(s) (expressed as percentage increase or decrease), (2) computing the change in mean intake from baseline to follow-up for the control group (expressed as percentage increase or decrease), and (3) computing the difference in the change between the intervention and control group (expressed as percentage increase or decrease). The difference in delta is based on the mean values reported in the studies we reviewed, and such statistics may have been originally reported as unadjusted or adjusted for covariates. Because we were performing secondary analyses on previously published articles, we had to use the means in the format in which the authors reported them.

Table 15 (adapted from Gorder et al.37), which reports results for total fat (percentage of energy), illustrates the technique we used. In this study, the intervention group reported a decrease of fat intake of 11.5 percent (from 38.2 percent of total daily energy intake at baseline to 33.8 percent of total energy intake at follow-up), whereas the control group reported a decrease of only 0.5 percent. The difference in the percentage change for the intervention and the control groups was 11.0 percentage points (11.5 -0.5 = 11.0 in absolute terms).

Table 15. Illustration of the Calculation of the Difference in Change in Total Fat Intake Between Groups.

Table

Table 15. Illustration of the Calculation of the Difference in Change in Total Fat Intake Between Groups.

An additional example (Table 16), which presents results for fruit and vegetable intake (among elementary school students) and includes multiple follow-up periods, is taken from a study by Reynolds et al.35 In this example, the intervention group reported a 51.7 percent increase in daily servings of fruits and vegetables at follow-up 1. Over the same period, the control group reported a decrease in fruit and vegetable intake (by 9.2 percent); hence, the difference in the percentage of change between the intervention and control groups at follow-up 1 was 60.9 percentage points (51.7 + 9.2 = 60.9). At follow-up 2, the intervention group reported an increase of 22.6 percent (from baseline); the control group reported a decrease of 12.0 percent. Thus, the difference in the rates of change between the groups at the second follow-up period was 34.6 percentage points (22.6 + 12.0 = 34.6).

Table 16. Illustration of the Calculation of the Difference in Percentage Change in Fruit and Vegetable Intake Between Groups at Multiple Follow-Up Periods.

Table

Table 16. Illustration of the Calculation of the Difference in Percentage Change in Fruit and Vegetable Intake Between Groups at Multiple Follow-Up Periods.

Our general approach was to calculate the differences in deltas for the selected outcomes for all eligible studies. We calculated this metric for all follow-up periods reported in the articles and for all eligible comparisons within an article. For example, if an article presented results for two intervention groups and a control group, we calculated the difference in delta for both intervention groups (i.e., intervention group 1 compared with the control group and intervention group 2 compared with the control group). Similarly, if an article reported results for separate population subgroups (e.g., males and females), we calculated the difference in delta for each group.

Because our primary goal was to derive a summary indicator of the average difference in deltas across all studies, and then among various groupings of studies (such as intervention settings or specific intervention features), we decided to use a single indicator of the difference in percentage change for each study at each reported follow-up period. This strategy prevented the results from a particular study from being counted more than once in the calculation of the overall difference in percentage change for all studies (or various groupings of studies).

In determining how to derive a single percentage change score for each article, we used the following criteria:

  • For studies reporting results for more than one intervention group, we used the results for the intervention group receiving the most intensive intervention (compared with the control group).
  • For studies reporting results separately for more than one measurement technique (e.g., reporting total fat as a percentage of energy intake derived from a food frequency questionnaire as well as reporting total fat as a percentage of energy intake derived from a four-day food record), we used the results for the most commonly used measurement technique (typically the food frequency questionnaire), regardless of the validity, reliability, or sensitivity to change of the technique.
  • For studies reporting results for population subgroups (e.g., males and females), we calculated a single, weighted mean (based on the proportion of the study population in the intervention and control groups) to use in the estimation of the difference in percentage change for the study.

Using these procedures, we developed a single percentage difference in change between intervention and control groups for each study at all reported follow-up periods. We used the differences in deltas calculated for individual studies to create summary measures indicating the effectiveness of interventions for various dietary outcomes. For example, we calculated summary measures indicating the effectiveness of interventions in general at reducing total fat intake or increasing fruit and vegetable intake based on the difference-in-deltas approach.

To present our results (including the overall intervention effect for various dietary outcomes, as well as for specific follow-up periods or among specific intervention settings or other characteristics), we chose the median difference in delta as the appropriate statistic to report because the distribution of differences in deltas calculated for individual studies was skewed. This analytic approach, in relying on a standardized metric, allowed us to compare the intervention effect across a variety of characteristics, such as intervention setting, the risk status of the study population, and specific intervention features. Therefore, for a particular dietary outcome, the median difference-in-deltas for one grouping of articles could be compared with the median difference-in-deltas for another grouping of articles. This in turn enabled us to draw conclusions about the efficacy of the intervention between the two groupings. In short, this technique was used to make relative comparisons about the success of different types of interventions at changing dietary behavior.

Additional analyses using this technique explored the "corroboration" of change in dietary intake with change in biochemical indicators. We calculated the median difference-in-deltas for biochemical outcomes (as well as the previously mentioned dietary outcomes) and then examined the correlation of differences-in-deltas for dietary outcomes with differences-in-deltas for biochemical outcomes. These analyses focused on blood lipids (specifically total blood cholesterol) because of the small number of studies measuring plasma carotenoids. Of the studies we reviewed, only one reported change in plasma carotenoids, 111 precluding our ability to draw conclusions about the corroboration of change in fruit and vegetable intake with biochemical markers.

Analysis of the Significance of the Intervention Effect

The final tier of our analysis approach involved a semiquantitative analysis that, in essence, simply summarized whether the investigators reported a statistically significant intervention effect. We performed this type of analysis for all outcomes in the Evidence Tables, and it included almost all of the articles we reviewed. The only criterion for including articles in this third group of analyses was that the statistical significance (at p < 0.05) of the intervention had to have been reported in the article. For each outcome reported in each article, we created a dichotomous indicator of whether the article reported a significant intervention effect.

Although many studies determined the significance of the intervention effect by multiple strategies, at multiple follow-up periods, or among multiple intervention groupings or population subgroups, we classified articles as having had a significant intervention effect if at least one of the results reported in the article was statistically significant. For example, if a study compared the mean fruit intake between an intervention and a control group at three follow-up periods, with results reported separately for males and females, and only the difference between means at follow-up 1 for females was statistically significant, the study would be classified as having had a significant intervention effect (for the outcome of fruit intake). Our classification of the significance of articles was based on the statistical tests that the investigators performed. These included a variety of approaches, such as the p value for parameter estimates for intervention status as a predictor, the group-by-time interaction effect in ANOVA models, t-tests for differences in means, chi-square tests for differences in proportions, and related statistics.

This analysis strategy is likely to overestimate the number of studies that truly observed a significant intervention effect. The reason is that among the studies we reviewed, frequently only p values for significant findings were reported. In addition, studies reporting significant results were more likely to have been submitted and published in the first place, resulting in publication bias. Although subject to some degree of bias, the "summary of significant findings" technique is advantageous because it enabled us to include unique or at least uncommonly reported outcomes that could not be analyzed with either of the previous strategies. In addition, with this approach we could include articles reporting insufficient statistical information (such as means) that otherwise could not be examined with either the meta-analysis or the difference-in-deltas approach. Finally, it accommodated the high degree of variability in various statistical procedures for determining the significance of the intervention effect that we observed in the articles in this report.

Using this technique, we classified all outcomes for each study as having had a significant effect or not. For simplicity in reporting and interpretation, we grouped the outcomes into broader categories. Fruit- and vegetable-related outcomes were classified into three categories:

  • Fruit and vegetable intake combined (including outcomes such as servings per day of fruits and vegetables, percentage of fruits and vegetables selected in relation to all foods selected, fruit and vegetable consumption scores)
  • Fruit intake (including outcomes such as servings per day of fruit, grams per day of fruit, fruit consumption scores, percentage of energy from fruit intake)
  • Vegetable intake (including outcomes such as servings per day of vegetables, grams per day of vegetables, portions per week of vegetables)
To accommodate the greater variety among dietary fat outcomes, we used five categories to group the various measures for dietary fat:
  • Intake of total fat (including outcomes such as percentage of total energy intake from fat, grams of fat)
  • Intake of saturated fat (including outcomes such as percentage of total energy intake from saturated fat, grams of saturated fat)
  • General fat intake scores or indices (including "fat scores" based on semiquantitative surveys)
  • Intake of individual high-fat foods or engagement in specific high-fat eating behaviors or practices (including general "high-fat practices" indices; specific high-fat behaviors such as using saturated fat for frying or eating poultry with the skin on; and intake of specific foods or types of foods such as high-fat meat, fried foods, butter or cream, or high-fat milk)
  • Intake of individual low-fat foods or engagement in specific low-fat eating behaviors or practices (including general "low-fat practices" indices or scales; specific low-fat behaviors such as trimming the fat from meat; and intake of specific foods or types of foods such as skim milk, low-fat spreads)

When analyzing how effective interventions were for the eight sets of dietary outcomes described above, we reported the proportion (and number) of articles reporting a significant intervention effect. This strategy enabled us to compare the proportion of one type of study (e.g., school-based interventions) reporting a significant intervention effect with the proportion of a different type of study (e.g., community interventions).

Relative Effectiveness of Interventions

Our primary goal (for each of the three secondary analysis strategies described above) was to determine the overall effectiveness of dietary interventions at changing dietary behavior. Our secondary goals included a determination of the relative effectiveness of different types of interventions and among different population subgroups at changing dietary behavior. For example, population characteristics have the potential to moderate the effect of behavioral interventions on dietary change. In addition, key intervention characteristics may influence the magnitude of dietary change. Intervention and population characteristics identified as being particularly important to explore as predictors of the efficacy of interventions included the following:

  • the age of the population
  • the risk status of the population
  • the intervention setting
  • the delivery mode of the intervention
  • the intervention intensity
  • the theoretical basis of intervention
  • the quality score of the article
  • whether the intervention included non-nutrition components
  • specific intervention components
    - family component
    - social support
    - small groups
    - interactive activities involving food
    - goal setting
    - cultural specificity
    - individual tailoring

We were able to explore only two population characteristics (age and risk status) because of the limited variability in study populations and the undetailed sample descriptions we encountered in our review of the articles. Although the consideration of gender, ethnicity, or socioeconomic status as moderating variables were a part of the original objectives of this report, we did not encounter sufficient variability in the studies we reviewed to explore these factors.

In our analyses, age indicated simply whether the study population included children (<14 years) or adults (>18 years); none of the studies we reviewed included youth (14 to 18 years) as the target of the intervention. Risk status indicated whether the study population was either at risk of or diagnosed with a particular disease (including CVD, type 2 diabetes, or cancer) or was not at risk.

We examined several intervention characteristics. The articles we reviewed were classified into one of four intervention settings: health care, school-based, worksite, or community/other. The delivery mode of the intervention indicates the role of the individual (or individuals) who delivered the intervention to the subjects. We identified four major delivery modes: self-administered, nonhealth professionals (including classroom teachers, physical education teachers, peers), health professionals (e.g., doctors, nurses), and nutritionists (including dietitians).

For each study in our review, we assigned one of three intensity levels to describe the total dose of the intervention: low, medium, or high. Factors considered in the assignment of the score included number, length, and duration of contacts (with individual contact considered more "intense" than group contacts); the existence of environmental-level interventions (such as alterations in cafeteria or vending machine options); and educational materials or media. Although our classifications were relatively subjective, they permitted us to determine the relative intensity of the interventions based on the limited information that was generally available. We refer to this characteristic as "intensity" in the remainder of the report, but we emphasize that our classification system was quite broad and included any aspects pertaining to the total dosage of the intervention.

The theoretical basis of the intervention involved a simple classification of whether the investigators used a theoretical framework in the design or implementation of the intervention. Because of the inconsistency and lack of detail in the articles we reviewed, this classification does not take into account the degree to which theory was used in the intervention. Rather, it is simply an admittedly crude indicator of whether the authors reported that a specific theory was used in the intervention (this was typically mentioned in either the introduction or the methods section of the articles, in the description of the intervention). Although we classified the articles as having or not having a theoretical basis, we recognize that the true use of theory is likely to be a continuum, rather than a dichotomy, and that in many articles it was extremely difficult to determine whether theory was actually used. In many cases, a strong theoretical base may have actually guided the intervention, but because of journal space limitations and editorial constraints, this fact may not have been evident.

In addition to classifying all articles on the basis of whether theory was used, we decided to explore the relationship between employing a theoretical framework and the significance of effects among recently published articles, given the increasing emphasis on incorporating theory in the design and implementation of behavioral interventions. Our rationale was that distinguishing between interventions that did or did not use theory at a time when awareness of the perceived need to use theory was substantially raised would provide a more appropriate comparison. Thus, we also classified the subset of articles published in or after 1995 as having or not having a theoretical basis.

The quality scores assigned to the articles (described in detail in a previous section of this chapter) ranged from 0 to 100. For use in the analyses, however, they were classified into three evenly distributed categories: low (scores <51), medium (scores ranging from 52 to 61), and high (scores >62). A dichotomous indicator, non-nutrition components, was created to reflect whether an intervention focused exclusively on modifying dietary behavior or if it included ancillary, non-nutrition components (such as physical activity modification, stress management, or smoking cessation).

Finally, we explored the influence of several specific behavioral change strategies in our analyses. Specifically, the articles we reviewed were coded as to whether the interventions included the following: a family component (such as family homework assignments, involving spouses in cooking classes), social support (including support groups), small group sessions, goal setting (and related self-monitoring components), interactive activities involving food (which includes intervention components such as taste testing, cooking classes), cultural or ethnic specificity (in which the intervention either was specifically designed for a particular cultural or ethnic group or included major components that were culturally or ethnically specific), and individual tailoring (which included specifically tailored nutritional messages generated through an interactive process with the intervention participants).

We did not have a large enough number of articles to explore interactions among the intervention and/or population characteristics described above (for instance, the impact of family components combined with goal setting). As a result, the influence of the components we explored cannot be conceptualized as independent effects. This fact, coupled with the likelihood that certain intervention components tended to be closely associated with one another (e.g., nutritionist-led interventions being more likely to incorporate interactive activities involving food), makes findings difficult to interpret. Some degree of confounding is inherent in our analysis plan because many characteristics may indeed be proxies for one another. This limitation should be kept in mind when interpreting the results presented in the following chapter.

We used the population and intervention characteristics described in this section as grouping variables in our three tiers of secondary analyses. For example, using the difference-in-deltas approach, we compared the median difference in change (between intervention and control groups) from one type of study (e.g., school-based interventions) with the median difference in change in another type of study (e.g., worksite interventions). Using the summary of significant effects approach, we compared the proportion of studies reporting significant findings in certain groupings of interest. Because many of the population and intervention characteristics were unevenly distributed among the 92 studies we analyzed (i.e., only a small number of articles employed social support components, making comparisons difficult to conduct), we had to establish minimum "cell sizes" for a particular comparison to be conducted in order to prevent extremely unstable estimates. Thus, we decided that a minimum of five studies had to be in each grouping before we would conduct a particular comparison.

Chapter 3 provides descriptive information about the studies we reviewed (based on the intervention and population characteristics desc ribed in this section). In addition, it presents the results of the three tiers of our secondary analyses.

Views

  • PubReader
  • Print View
  • Cite this Page

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...