Overview

This evidence report on omega-3 fatty acids and cardiovascular disease (CVD) outcomes is based on a systematic review of the literature. To identify the specific issues central to this report, the Tufts-New England Medical Center (NEMC) Evidence-based Practice Center (EPC) held meetings and teleconferences with a Technical Expert Panel (TEP). A comprehensive search of the medical literature was conducted to identify studies addressing key questions. Evidence tables of study characteristics and results were compiled, and the methodological quality and applicability of the studies were appraised. Study results were summarized with qualitative reviews of the evidence, summary tables, and quantitative meta-analyses, as appropriate.

Several individuals and groups collaborated with the Tufts-NEMC EPC in preparing this report. The TEP served as our science partner. The EPC engaged technical experts and representatives from the Agency for Healthcare Research and Quality (AHRQ) and the National Heart, Lung, and Blood Institute (NHLBI) to help refine key questions, identify important issues, and define parameters to the report. The Tufts-NEMC EPC also worked in conjunction with the EPCs at the University of Ottawa (UO) and Southern California-RAND (SC-RAND). Together, the 3 EPCs will produce evidence reports on 10 topics related to omega-3 fatty acids over a 2-year period. The 3 EPCs coordinated activities with the goal of producing evidence reports of uniform format. Through frequent teleconferences and email contact, approaches toward data presentation, summary and evidence table layout, and study quality and applicability assessment were standardized, whenever feasible. In addition, the primary literature searches for all evidence reports were performed by the UO EPC, using identical search terms for studies of omega-3 fatty acids. However, each EPC developed its own eligibility criteria to identify relevant studies as appropriate for its topic.

Analytic Framework

To guide our assessment of studies that examine the association between omega-3 fatty acids and cardiovascular outcomes, we developed an analytic framework that maps the specific linkages associating the populations of interest, the exposures, modifying factors, and outcomes of interest (Figure 2.1) 21. The framework graphically presents the key components of well-formulated study questions:

Figure 2.1 Analytic framework for omega-3 fatty acid exposure and cardiovascular disease. This framework concerns the effect of omega-3 fatty acid exposure (as a supplement or from food sources) on cardiovascular disease. Populations of interest are noted in the top rectangle, exposure in the oval, outcomes in the rounded rectangles, and effect modifiers in the hexagon. Thick connecting lines indicate associations and effects reviewed in this and the accompanying report. Lists noted in a smaller font indicate the specific factors reviewed. CVD indicates cardiovascular disease; FA, fatty acid; RBC, red blood cell (erythrocyte); WBC, white blood cell (leukocyte).

Figure

Figure 2.1 Analytic framework for omega-3 fatty acid exposure and cardiovascular disease. This framework concerns the effect of omega-3 fatty acid exposure (as a supplement or from food sources) on cardiovascular disease. Populations of interest are noted (more...)

  1. Who are the participants (i.e., what is the population and setting of interest, including the diseases or conditions of interest)?
  2. What are the interventions?
  3. What are the outcomes of interest (intermediate and health outcomes)?
  4. What study designs are of value?

Specifically, this analytic framework depicts the chain of logic that evidence must support to link the intervention (exposure to omega-3 fatty acids) to improved health outcomes.

This report and the accompanying report, Effects of Omega-3 Fatty Acids on Cardiovascular Risk Factors and Intermediate Markers of Cardiovascular Disease, review the evidence addressing the associations or effects of omega-3 fatty acids in humans. Specifically, this report examines evidence addressing the association between omega-3 fatty acids and clinical cardiovascular outcomes, their efficacy in improving CVD outcomes, and potential adverse effects of omega-3 fatty acid intake in humans. The accompanying report examines evidence addressing both the association in humans between omega-3 fatty acids and cardiovascular intermediate outcomes or risk factors and the association between omega-3 fatty acids and tissue or plasma levels of omega-3 fatty acids.

In both reports, the 3 specific populations of interest are: (1) healthy adults with no known CVD or risk factors; (2) adults at increased risk of CVD due specifically to diabetes, hypertension, or hyperlipidemia; and (3) adults with known CVD. The exposure of interest is omega-3 fatty acids. Unlike medications, there are numerous possible sources, types, and possible dosages for omega-3 fatty acids. Thus, questions of interest include how different sources, dosages, and relative proportions of the fatty acids differ in their effects on the outcomes of interest. Included are questions addressing possible differences between the effects of supplements (e.g., fish oil capsules) and dietary sources (e.g., fatty fish), the effect of duration of intervention or exposure, and whether any effect is sustained after stopping treatment.

The analytic framework does not directly address the level of evidence that is necessary to evaluate each of the effects. Large randomized controlled trials that are adequately blinded and otherwise free of substantial bias provide the best evidence to prove causation between intervention and outcome. However, this study design is not always available (or possible). Observational studies provide lesser degrees of evidence that are usually hypothesis-generating regarding causation. The current analysis relies as much as possible on high quality, randomized controlled trials, using evidence from observational studies when data are relatively sparse.

Key Questions Addressed in this Report

The purpose of this evidence report is to summarize information from studies that address specific key questions. One general question concerns the intake of omega-3 fatty acids in the US population, and 3 additional questions address the relationship between omega-3 fatty acids and CVD. CVD question 1 pertains to the clinical effects of omega-3 fatty acids on clinical CVD outcomes; CVD question 2 evaluates the relative effects of the numerous sources, compositions, dosages, and uses of omega-3 fatty acids and related factors; and CVD question 3 pertains primarily to the association between omega-3 fatty acids and adverse events and drug interactions. The key questions and their related sub-questions are outlined in detail below.

General Question

What are the mean and median intakes of eicosapentaenoic acid (EPA, 20:5 n-3), docosahexaenoic acid (DHA, 22:6 n-3), alpha linolenic acid (ALA, 18:3 n-3), fish, fish oil, and omega-6 fatty acids, and what is the mean and median omega-6 to omega-3 fatty acid ratio, in the US population?

  • Do consumption levels differ among subpopulations?

CVD Questions

What is the efficacy or association of omega-3 fatty acids (DHA, EPA or ALA supplements, and fish consumption) in reducing CVD events (including all-cause mortality, CVD mortality, non-fatal CVD events, and new diagnosis of CVD)?

  • What is the efficacy or association of omega-3 fatty acids in preventing incident CVD outcomes in people without known CVD (primary prevention) and with known CVD (secondary prevention)?
  • How does the efficacy or association of omega-3 fatty acids in preventing incident CVD outcomes differ in sub-populations, including men, pre-menopausal women, post-menopausal women, and different age groups?
  • What are the effects of potential confounders — such as lipid levels, body mass index (BMI), blood pressure, diabetes, aspirin use, hormone replacement therapy, and cardiovascular drugs — on associations found in prospective cohort studies?
  • What is the relative efficacy of omega-3 fatty acids on different CVD outcomes? Can the CVD outcomes be ordered by strength of treatment effect of omega-3 fatty acids?

Omega-3 fatty acid variables and modifiers:

  • What is the efficacy or association of specific omega-3 fatty acids (DHA, EPA, ALA), and different ratios of omega-3 fatty acid components in dietary supplements, on CVD outcomes?
  • Does the ratio of omega-6 to omega-3 fatty acid intake affect the efficacy or association of omega-3 fatty acid intake on CVD outcomes?
  • How does the efficacy or association of omega-3 fatty acids on CVD outcomes differ by source (e.g., dietary fish, dietary oils, dietary plants, fish oil supplement, flax seed supplement)?
  • How does the efficacy or association of omega-3 fatty acids on CVD outcomes differ by different ratios of DHA, EPA, and ALA?
  • Is there a threshold or dose-response relationship between omega-3 fatty acids and CVD outcomes?
  • How does the duration of intervention or exposure affect the treatment effect of omega-3 fatty acids on CVD outcomes?
  • Are treatment effects or the association of omega-3 fatty acids on CVD events sustained after the intervention or exposure stops?
  • What is the effect or association of baseline dietary intake of omega-3 fatty acids on the efficacy of omega-3 fatty acid supplements on CVD outcomes?
  • Does the use of medications for CVD and/or CVD risk factors (including lipid lowering agents and diabetes medications) affect the efficacy or association of omega-3 fatty acids?

Adverse events and drug interactions:

  • What adverse events related to omega-3 fatty acid dietary supplements are reported in studies of CVD outcomes and markers?
  • What adverse events related to omega-3 fatty acid dietary supplements are reported specifically among diabetics and people with CVD in studies of CVD outcomes and markers?
  • What interactions between omega-3 fatty acid dietary supplements and medications are reported in studies of CVD outcomes and markers?
  • What interactions between omega-3 fatty acid dietary supplements and medications are reported specifically among diabetics and people with CVD in studies of CVD outcomes and markers?

Method to Assess the Dietary Intake of Omega-3 Fatty Acids in the US population

Two major sources of dietary intake data in the US population are the Continuing Survey of Food Intakes by Individuals (CSFII) conducted by the US Department of Agriculture (USDA) and the National Health and Nutrition Examination Survey (NHANES) conducted by the National Center for Health Statistics (NCHS). The USDA's most recent survey, the CSFII 1994-96, popularly known as the What We Eat in America survey, addressed the requirements of the National Nutrition Monitoring and Related Research Act of 1990 (Public Law [P.L.] 101–445) for continuous monitoring of the dietary status of the American population 22. In CSFII 1994-96, improved data collection methods (i.e., the multiple-pass approach for the 24-hour recall) were used. Given the normal, large day-to-day variation in dietary intake, multiple 24-hour recalls are considered to be best suited for most nutrition monitoring 9 and produce stable estimates of mean nutrient intakes from groups of individuals.

The NHANES is designed to collect periodic information on the dietary, nutritional, and health status of the civilian, non-institutional US population. Since 1970, 3 NHANES have been completed: NHANES I, 1971-74; NHANES II, 1976-80; and NHANES III, 1988-94. NHANES is unique in that it combines a home interview with health tests that are done in a Mobile Examination Center (MEC). The Third National Health and Nutrition Examination Survey (NHANES III, 1988-94) was conducted at 89 locations in the US. Data obtained through the survey include dietary intake (one 24-hour recall and food frequency questionnaire), socioeconomic and demographic information, biochemical analyses of blood and urine, physical health behaviors, and health conditions. Although multiple 24-hour recalls are considered the “gold standard” for nutrition monitoring (e.g., the dietary assessment method used in CSFII, 1994-96), single 24-hour recalls will also produce reasonably accurate estimates of mean nutrient intakes if the sample size is large23. By combining dietary data from NHANES III with its unique MEC health test results, we were able to analyze the mean intake of omega-3 fatty acids among people with and without cardiovascular diseases, an analysis that could not be performed if we used CSFII data.

The 3rd National Health and Nutrition Survey (NHANES III) Database

The NHANES III, 1988-94 database was used to examine the population intake of omega-3 fatty acids in the US (General Question). NHANES III was designed to collect information on the US population aged = 2 months. Mexican Americans and non-Hispanic African Americans, children = 5 years old, and adults = 60 years old were over-sampled to produce more precise estimates for these population groups. There were no imputations for missing 24-hour dietary recall data. A total of 29,105 participants had complete and reliable dietary recall.

Definitions of Key Variables

The population means and standard errors of the mean (SEM) of total polyunsaturated fatty acids (PUFAs), ALA, EPA, and DHA by sex, age, and/or income levels have been presented in a report by the National Center for Health Statistics 2. However, the sub-population grouping system is different from the system that is used in Institute of Medicine (IOM) reports. In order to provide the most parsimonious interpretation of IOM reports and this evidence report, we have decided to adopt the approach used in Dietary References Intakes (DRIs) published by the IOM 2. The main variables in this evidence report are defined as follows:

  • Age groups: Subjects' age in months was used to form ten age groups: 2–6 months, 7–12 months, 1–3 years, 4–8 years, 9–13 years, 14–18 years, 19–30 years, 31–50 years, 51–70 years, and 71+ years. Age in months was calculated by computing the number of months between the screener questionnaire date and each subject's date of birth. Two additional age groups were created for the adult sub-population: less than 45 years old, and 45 years old and older.
  • Race/ethnicity groups: Four ethnicity groups were used in this report: non-Hispanic white, non-Hispanic black, Mexican American, and others. The groups were defined by the race or ethnicity reported by respondents. Respondents were asked to identify themselves as: black; Mexican or Mexican American; white, non-Hispanic; Asian or Pacific Islander; Aleut, Eskimo, or American Indian; or other Latin American or other Spanish.
  • Poverty: Two poverty income ratio (PIR) groups were created for use in analyses: PIR = 1.3 and poverty income ratio > 1.3. The numerator of the ratio was the midpoint of the respondent's family income category. The denominator was based on the poverty threshold, the respondent's age, and the calendar year of the interview.
  • Urbanization: Metropolitan or non-metropolitan areas were based on the USDA's rural-urban codes that categorize counties by degree of urbanization and nearness to a metropolitan area.
  • People with a history of CVD: Respondents defined in this report as having a history of CVD were those who responded “yes” to one of the following interview questions: (1) Has a doctor ever told you that you had congestive heart failure? (2) Has a doctor ever told you that you had a stroke? (3) Has a doctor ever told you that you had a heart attack? Respondents whose electrocardiography results showed a probable or possible myocardial infarction (MI), or probable or possible left-ventricular hypertrophy (LVH), by the Minnesota Code (Appendix C) were also defined as having CVD.
  • Polyunsaturated fatty acids: ALA, EPA, DHA, docosapentaenoic acid (DPA, 22:5 n-3), and linoleic acid (LA, 18:2 n-6) data, estimated from a single 24-hour dietary recall, were used.

Analyses of NHANES III Data

The data were analyzed using SAS-callable SUDAAN, version 7.5.6 (Research Triangle Institute, Research Triangle Park, NC), which is a statistical analytic software program that adjusts for the complex NHANES III sample design. All analyses incorporated sampling weights that adjusted for unequal sampling probabilities. Variance estimations were made with the WR method (sampling With Replacement). Each denominator has 49 degrees of freedom. The design effect (deff4) was defined as the ratio of the properly computed actual variance of an estimated parameter to the variance based on a simple random sample of the same size.

We used simple linear regression to test the significance of the differences in daily intake of PUFAs between groups. The adjusted means for categorical covariates in the regression model were calculated with the least squares method. Statistical significance of the correlation between the dependent variables (e.g., intake of ALA) and independent variables (e.g., sex groups, age groups, CVD groups) were calculated with the Wald chi-square statistics. The details of these statistical methods are described in the SUDAAN user's manual. Since the amount of dietary PUFAs may be associated with the amount of dietary total fat, results expressed as grams per day can be misleading. Thus, all PUFAs used in the tests of significant differences between groups were measured as percent of total energy intake per day (% kcal/day).

All analyses assume a normal distribution of the nutrient intake. However, data related to EPA and DHA are very skewed. As a result, the mean and SEM estimates for these nutrients should be used and interpreted with caution. The reliability of an estimated mean or median also depends on the coefficient of variation or relative standard error (RSE), defined as the ratio between the standard error of the estimate and the estimate, multiplied by 100. Estimates with an RSE greater than 20 percent are deemed unreliable in this report.

Literature Search Strategy

A comprehensive literature search was conducted to address the 3 key questions related to CVD. Relevant studies were identified primarily through search strategies conducted in collaboration with the UO EPC. The Tufts-NEMC EPC, using the Ovid search engine, conducted preliminary searches on the Medline database. The final searches used six databases including Medline from 1966 to week 2 of February 2003, PreMedline February 7, 2003, Embase from 1980 to week 6 of 2003, Cochrane Central Register of Controlled Trials 4th quarter of 2002, Biological Abstracts 1990 - December 2002, and Commonwealth Agricultural Bureau (CAB) Health from 1973 to December 2002. Subject headings and text words were selected so that the same set could be applied to each of the different databases with their varying attributes. Supplemental search strategies were conducted as needed. Additional publications were referred to us by the TEP and the other 2 EPCs. Details about selected terms used in the search strategy are discussed below.

Omega-3 Fatty Acids Search Strategy

A wide variety of search terms were used to capture the many potential sources of omega-3 fatty acids. Search terms used include the specific fatty acids, fish and other marine oils, and specific plant oils (flaxseed, linseed, rapeseed, canola, soy, walnut, mustard seed, butternut, and pumpkin seed). These terms were used in all search strategies. Because some studies evaluated the effect of nuts on CVD outcomes without specifying in the abstract the type of nuts used in the study, we performed a supplemental Medline search using the term “nut” as a text word for studies of CVD.

Cardiovascular Search Strategy

The primary search strategy was designed to address both the clinical and intermediate outcomes of CVD in humans (Appendix A). In order to identify CVD outcomes in human studies, the search was divided into 3 categories consisting of controlled trials, other studies, and reviews. These 3 categories were further divided into English and non-English subsets. To address the questions regarding stroke, the Tufts-NEMC EPC performed a separate search on the Medline database. This search yielded no additional relevant publications.

Diabetes

Because specific terms referring to diabetes had been omitted from the primary search strategy, a supplemental search strategy was conducted on March 29, 2003. The diabetes supplemental search strategy included relevant search terms for diabetes. This search strategy resulted in an additional 410 citations for screening.

Overall

The final number of citations identified by the database searches is approximate. Because the 5 main databases used in the search employ different citation formats, duplicate publications were encountered. The UO EPC eliminated most of the duplicate publications; however, because of many different permutations, it was impossible to identify all of them. We eliminated additional duplicate publications as we encountered them.

Ongoing automatic updates of Medline searches were conducted using the CVD search strategy. The last automatic update was on April 19, 2003. The UO EPC conducted a final update search of the other databases on April 10, 2003.

Study Selection

Abstract Screening

All abstracts identified through the literature search were screened using eligibility criteria developed in conjunction with the TEP. These criteria were designed to minimize incorrect exclusion of relevant studies. We included all English language original, experimental, or observational studies that evaluated any potential source of omega-3 fatty acids in at least 5 human subjects, regardless of the study outcomes reported in the abstract. In addition, we excluded abstracts that clearly included only subjects who had a non-CVD-related condition (e.g., cancer, schizophrenia, or organ transplant). Reports published only as letters or as abstracts in proceedings were also excluded. All abstracts were categorized to 1 or more of the key questions or as rejects.

Full Article Inclusion Criteria

Articles that passed the abstract screening process were retrieved, and the full articles were screened for eligibility. The following types of articles were rejected during this round: review articles, inappropriate human population, pediatric studies and studies conducted on subjects less than 19 years old, no mention of omega-3 fatty acid intake, dietary supplements, or fish consumption, daily dose of omega-3 fatty acid greater than 6 g, fewer than 5 subjects in omega-3 fatty acid arm(s), prospective interventional studies of less than 4 weeks duration, and no appropriate outcome of interest reported. Studies that reported only the tissue level of omega-3 fatty acid without explicitly reporting the amount of omega-3 fatty acid consumed were also excluded. However, we included studies of Mediterranean diets and studies that reported fish servings. Specific sources of omega-3 fatty acid considered acceptable included fish oils, dietary fish, canola (rapeseed) oil, soybean oil, flaxseed or linseed oil, walnuts or walnut oil, and mustard seed oil. Other sources were eligible if omega-3 fatty acid levels were reported to be greater than control. For each study that was rejected, the reason(s) for rejection was noted. For analyses of adverse events and drug interactions, all studies were included regardless of omega-3 fatty acid dose or study duration (including washout period).

Inclusion and exclusion criteria for maximal omega-3 fatty acid intake were based on discussions with the TEP, in which it was agreed that omega-3 fatty acid intake above 6 g per day is impractical and has little relevance for health care recommendations. Therefore, with the exception of studies of adverse events, the inclusion criterion for maximum daily intake was set at 6 g per day and studies of higher daily intake were excluded. The definition of omega-3 fatty acid dose varied greatly across studies. Thus, the maximal allowable dose may have applied to total daily omega-3 fatty acid, total EPA+DHA, or a total of other combinations of omega-3 fatty acids. The total did not refer to total fish oil.

In this report, we accepted randomized controlled trials (RCTs) or prospective cohort studies with a minimum of 1-year follow-up to address CVD outcome questions. We also accepted case-control studies and cross-sectional studies that assessed the prevalence of CVD in populations with varying levels of omega-3 fatty acid consumption. In some cases, a study was reported in multiple publications (e.g., interim results might have been reported in 1 publication and various outcomes in others). For these studies, we identified and grouped articles belonging to the same overall study and used data from the latest publication, supplemented by data from earlier publications, as appropriate.

Selection of Studies for Adverse Events and Drug Interactions

Human studies that were analyzed for clinical outcomes (for this report) or for risk factors (for the accompanying report, Effects of Omega-3 Fatty Acids on Cardiovascular Disease Risk Factors) were reviewed for data on adverse events and drug interactions. The eligibility criteria for these analyses were broader than for analyses of CVD outcomes, as described above.

The Food and Drug Administration's (FDA) definition of adverse events was used [FDA]. This definition includes morbidity, mortality, and evidence of organ damage. Because fishy after-taste is almost universally reported in subjects taking fish oil supplements24 it was explicitly excluded as an adverse event in this report.

Analyses of data on adverse events were limited to fish oils or omega-3 fatty acid supplements. Food-related illnesses and toxicities due to marine food sources, cooking oils, and cooking methods are beyond the purview of this report. Thus, data on mercury toxicity and carcinogenic hydrocarbons from grilling were not reviewed.

We looked for studies that evaluated potential interactions between omega-3 fatty acid supplements and commonly used drugs including, but not limited to, hormone replacement therapy, diabetes medications, aspirin, and anticoagulants. In the studies that reported serious adverse events such as clinical bleeding, we note the concurrent medications that the subjects were taking.

Data Extraction Process

We developed an electronic data form to collect the data extracted from studies for this report. In an iterative process, the data form underwent modifications and data extractors underwent training and consensus building. Consensus was reached on definitions, and issues specific to omega-3 fatty acid studies were addressed. After this process, each study was screened for eligibility criteria and for outcomes using the electronic form. Each eligible study was then fully extracted by a single reviewer. Data extraction problems were addressed during weekly meetings. Occasional sections were re-extracted to ensure that uniform definitions were applied across extracted studies. Problems and corrections were noted through spot checks of extracted data and during the creation of summary and evidence tables. A second reviewer independently verified the data in the summary tables using the original article.

Items extracted included: factors related to study design (randomization method, allocation concealment method, blinding, study duration, and funding source), population characteristics (country, eligibility criteria, demographics, comorbid conditions, concomitant medications, and baseline diet), interventions and comparison groups (description of omega-3 fatty acid and control interventions or diets, including amount of specific fatty acids), outcomes of interest (number enrolled and analyzed, intermediate and clinical outcomes, adverse events, reasons for withdrawals, results [including baseline value, final value, within-treatment change or between-treatment difference, and variance, as reported]), and whether each study addressed each of the key questions. In addition, each study was categorized based on applicability and study quality as described below.

Grading Evidence

Studies accepted in evidence reports have been designed, conducted, analyzed, and reported with various degrees of methodological rigor and completeness. Deficiencies in any of these processes may lead to biased reporting or interpretation of the results. While it is desirable to grade individual studies so that readers of evidence reports are informed about the degree of potential bias, grading the quality of evidence is not a straightforward process even for a single type of study design. For example, despite many attempts, most factors commonly used in the quality assessment of RCTs have not been found to be consistently related to the direction or magnitude of the reported effect size 25. There is still no uniform approach to reliably grade published studies based on the information reported in the literature. As a result, different EPCs have used a variety of approaches to grade study quality in past evidence reports.

To evaluate the quality of studies included in this report, we first assessed each study against criteria specific to its study design (RCT, prospective cohort study, case control study). Based on this assessment, we then assigned a summary quality grade that grades each study within its particular study design strata.

In this section, we discuss quality rating criteria for each type of study design and our summary quality rating system. We also discuss how we assessed a study's applicability, sample size, and results.

Quality Rating Criteria for Randomized Controlled Trials

As part of the overall omega-3 fatty acid project, the 3 collaborating EPCs agreed to use the Jadad Score and adequacy of random allocation concealment as elements to grade individual randomized controlled trials 26, 27. We also agreed that individual EPCs might add other elements to this core set, as we deemed appropriate. All EPCs agreed that studies should not be graded using a single numerical quality score, as this has been found to be unreliable and arbitrary 28.

The Jadad Score assesses the quality of RCTs using 3 criteria: adequacy of randomization, double blinding, and dropouts 26. A study that meets all 3 criteria gets a maximum score of 5 points. Adequacy of random allocation concealment was assessed as adequate, inadequate, or unclear using criteria described by Schultz et al 27.

The Jadad and Schulz scores address only some aspects of the methodological quality of RCTs. In particular, items in the core set ignore potential biases due to analytic and reporting problems in a study. To rectify this, we also assessed each RCT for the following:

  • Validity of methods used to assess diet
  • Errors or discrepancies in reporting results

Quality Rating Criteria for Prospective Cohort Studies

Unlike RCTs, where there is at least some empirical evidence to support the use of the core set of quality rating items, there is no empirical data to support the use of elements that should comprise a core set for non-randomized studies such as cohort and case-control studies. Because prospective cohort and case control studies do not have randomization, allocation concealment, and blinding, a core set different from that used for RCTs must be defined for these types of studies. In addition, because this report focuses on the effect of omega-3 fatty acids on CVD, the studies must estimate the amount of omega-3 fatty acid consumed by the study population as accurately as possible. We used the following criteria to assess the quality of prospective cohort studies:

  • Unbiased selection of the cohort (prospective recruitment of subjects)
  • Sufficiently large sample size (>1,000 subjects)
  • Adequate description of the cohort
  • Use of validated dietary assessment method
  • Quantification of the type and amount of fish/estimates of omega-3 fatty acid intake
  • Use of validated method for ascertaining clinical outcomes
  • Adequate follow-up period (at least 5 years)
  • Completeness of follow-up
  • Analysis (multivariate adjustments) and reporting of results

Quality Rating Criteria for Case Control Studies

Criteria used to assess the quality of case control studies include:

  • Valid ascertainment of cases
  • Unbiased selection of cases
  • Appropriateness of the control population
  • Verification that the control is free of CVD
  • Comparability of cases and controls with respect to potential confounders
  • Validated dietary assessment method
  • Appropriateness of statistical analyses

Generic Summary Quality Grade for All Studies

After evaluating each study against its design-specific quality criteria, we applied a 3 category (A, B, C) summary quality grading system that we have used in most of our previous EPC evidence reports, as well as in several evidence-based clinical practice 29. This scheme defines a generic grading system for study quality that is applicable to each type of study design (i.e., RCT, cohort study, case-control study). The categories are defined as follows:

  1. Least bias; results are valid. A study that mostly adheres to the commonly held concepts of high quality, including the following: a formal randomized study; clear description of the population, setting, interventions, and comparison groups; appropriate measurement of outcomes; appropriate statistical and analytic methods and reporting; no reporting errors; less than 20% dropout; clear reporting of dropouts; and no obvious bias.
  2. Susceptible to some bias, but not sufficient to invalidate the results. A study that does not meet all the criteria in category A. It has some deficiencies but none likely to cause major bias. Study may be missing information making assessment of the limitations and potential problems difficult.
  3. Significant bias that may invalidate the results. A study with serious errors in design, analysis, or reporting. These studies may have large amounts of missing information or discrepancies in reporting.

The summary quality grading system evaluates and grades the studies within each of the study design strata. By design, it does not attempt to assess the comparative validity of studies across different design strata. Thus, in interpreting the methodological quality of a study, one should note the study design and the quality grade that it received. For RCTs, in addition to the summary quality grade, we also indicate the Jadad score and the rating of the adequacy of allocation concealment.

While it might be desirable to rank the quality of all studies on the same scale regardless of study design, experience with this approach is limited and has never been validated. In fact, using a single rating scale for all studies creates potential problems. For example, a hierarchy of study design that places RCTs above cohort studies in terms of methodological rigor is commonly accepted. However, if an RCT is seriously flawed, the results may be more biased than a well-done cohort study.

Applicability

Applicability addresses the relevance of a given study to a population of interest. Every study applies certain eligibility criteria when selecting study subjects. Most of these criteria are explicitly stated (e.g., disease status, age, sex). Some may be implicit or due to unintentional biases, such as those related to study country, location (e.g., community vs. specialty clinic), or factors resulting in study withdrawals. The question of whether a study is applicable to a population of interest (such as Americans) is distinct from the question of the study's methodological quality. For example, due to differences in the background diets, an excellent study of Japanese men may be very applicable to people in Japan, but less applicable to Japanese American men, and even less applicable to African American men. The applicability of a study is thus dictated by the questions and populations that are of interest to those analyzing the studies.

In this report, the focus is on the US population and on specific subgroups within that population (i.e., healthy Americans, Americans with CVD, and Americans with diabetes or dyslipidemia), as specified in the scope of work for this series of evidence reports. To capture the potential applicability of studies to the different populations of interest as defined in the scope of work, we define the following target population categories:

GEN General population. Typical healthy people similar to Americans without known CVD.

CVD Cardiovascular disease population. Subjects with a history of, or currently with, 1 of the following: stroke, myocardial infarction, angina, ischemic peripheral vascular disease, or other condition as defined by the author.

We planned to include categories for diabetic and dyslipidemic populations but found no relevant studies within these categories.

Even though a study may focus on a specific target population, limited study size, eligibility criteria, and the patient recruitment process may result in a narrow population sample that is of limited applicability, even to the target population. To address this issue, we categorized studies within a target population into 1 of 3 levels (I, II, III) of applicability that are defined as follows:

  1. Sample is representative of the target population. It should be sufficiently large to cover both sexes, a wide age range, and other important features of the target population (e.g., diet).
  2. Sample is representative of a relevant sub-group of the target population, but not the entire population. For example, while the Nurses Health Study is the largest such study and the results are highly applicable to women, it is nonetheless representative only of women. A fish oil study in Japan, where the background diet is very different from that of the US, also falls into this category.
  3. Sample is representative of a narrow subgroup of subjects only, and is of limited applicability to other subgroups. For example, a study of the oldest-old men or a study of a population on a highly controlled diet.

In the summary tables, each study receives a combined applicability grade comprised of the target population (GEN or CVD) and the 3-level grade (I, II, III). For example, GEN-I represents a study of subjects representative of the general population in the US, such as a study of the NHANES population. Studies such as the Nurses Health Study and the Health Professionals Study are graded GEN-II because of each study's focus on a single sex. If several studies of complementary populations (e.g., the Nurses Health Study and the Health Professionals Study) were viewed together, they would offer highly applicable evidence for the general population and receive a grade of GEN-I.

Sample Size

The study sample size provides a quantitative measure of the weight of the evidence. In general, large studies provide more precise estimates of efficacy and associations. In addition, large studies are more likely to be generalizable; however, large size alone does not guarantee broad applicability.

Results of Randomized Clinical Trials

RCTs typically report a relative risk or the number of events for the outcome of interest. When relative risk was reported, we calculated it along with the confidence interval to verify the accuracy of the reporting. We also calculated it when only the number of events was reported. We present the adjusted relative risks when these were reported.

Results of Observational Studies

Prospective cohort studies typically categorize subjects into different quantiles (e.g., tertiles, quartiles, quintiles) of omega-3 fatty acid or fish intake and report the associated relative risk for the outcome of interest. For studies that report both unadjusted and multivariate adjusted results, we report the adjusted results in the evidence and summary tables.

Due to the heterogeneous nature of the studies (e.g., different population, background diet, dietary assessment method, and methods used to report estimates of fish or omega-3 fatty acid intake), meta-analyses were not feasible for this group of studies. To succinctly report each study's results and to help readers interpret them, we created a qualitative score or “overall effect” metric to supplement the main quantitative results in the summary tables. The overall effect metric is defined as follows:

+ + Clinically meaningful benefit demonstrated. Study reported on the clinical outcome of interest in 1 or both of the following ways:

  • statistically significant trend of benefit for the quantile estimates of fish/omega-3 fatty acid intake
  • at least one-half of the quantile estimates of fish/omega-3 fatty acid intake reported statistically significant beneficial effects of at least a 10% relative risk (RR) reduction (i.e., RR < 0.9), and no quantile reported a statistically significant adverse outcome

+ A clinically meaningful beneficial trend exists but is not conclusive. Study reported on the clinical outcome of interest in 1 or both of the following ways:

  • a borderline significant (0.10 > P > 0.05) trend of benefit for the quantile estimates of fish/omega-3 fatty acid intake
  • non-significant but potentially clinically meaningful effect (RR <0.9) in at last one-half of the quantile estimates, and no quantile reported a statistically significant adverse outcome

0 Clinically meaningful effect not demonstrated or is unlikely. Study reported clinically unimportant differences between low/no fish intake with various higher levels of fish intake. The majority of the quantiles of estimates of fish/omega-3 fatty acid intake reported less than 10% relative difference compared with the reference (i.e., 1.1>RR>0.9)

- Harmful effect demonstrated or is likely. Study reported on the clinical outcome of interest in one or both of the following ways:

  • a positive association (P<.10) between quantile estimates of fish/omega-3 fatty acid intake and increased risk
  • several quantile estimates reported RR >1.1

Evidence Reporting Format

Evidence and Summary Tables

We report the evidence in 3 complementary forms:

  1. Evidence tables offer a detailed description of the studies we identified that address each of the key questions. These tables provide detailed information about the study design, patient characteristics, inclusion and exclusion criteria, interventions and comparators evaluated, and outcomes. A study, regardless of how many interventions or outcomes were reported, appears once in the evidence tables. Evidence tables are grouped into RCTs and observational studies (cohorts, case-control, cross-sectional). Within each group, the studies are ordered alphabetically by the first author's last name to allow for easy searching within the tables.
  2. Summary tables succinctly report on each study using summary measures of the main outcomes. These tables were developed by condensing information from the evidence tables and are designed to facilitate comparisons and synthesis across studies. Summary tables include important concise information regarding study size, intervention and control, study population (e.g., general population or CVD), outcome measures, methodological quality, and applicability. A study with multiple populations, methods of reporting estimates of omega-3 fatty acid intake, or clinical outcomes may appear multiple times in different summary tables. Because there were few RCTs and almost as many outcomes to report, we organized the RCTs into 2 groups (trials of omega-3 fatty acid supplements and trials of diet or dietary advice) to reduce the number of tables and minimize redundant information.
    Summary tables for prospective cohort and case-control studies were organized based on clinical outcomes. For each of the clinical outcomes is a table for estimates of omega-3 fatty acid consumption and a table for estimates of fish consumption. Within each table, cohort studies preceded case-control studies and studies are ordered by the number of study subjects.
  3. Summary matrices provide an alternative to meta-analysis (when meta-analysis is not feasible) to facilitate the synthesis of a body of evidence. A summary matrix organizes potentially disparate studies into more homogeneous subgroups by their methodological quality and applicability grades. This allows the reader to appreciate the number of studies available and the effect size of these studies. Because there were too few RCTs and too few cohort studies of the CVD population, summary matrices were created only for prospective cohort studies for the general population in this report. Each summary matrix has applicability grades as row headings and methodological quality grades as column headings. Thus, 3 applicability grades and 3 methodological quality grades create a matrix with 9 cells. Studies assessed with a specific combination of methodological and applicability grades are displayed in their respective cells. Information displayed includes study name, study size, a measure of the effect size, and other information that may help to interpret the results.

Adverse Events Reporting

Separate adverse events evidence tables were not created. Most of these studies were included in the evidence tables of RCTs in this report or in the accompanying risk factor report. In this report, we produced summary tables on adverse events for two categories of studies: RCTs or crossover studies that compared an omega-3 fatty acid supplement with a control, and single arm cohort studies. For RCTs, we report the number and percentage of adverse events for both the omega-3 fatty acid arm and control arms for the following categories: clinical bleeding (nasal, hematuria, gastrointestinal, and other bleeding), gastrointestinal complaints, diarrhea, headaches, and withdrawals due to adverse events. We noted the dosages of omega-3 fatty acid and the control, as well as the study duration and the number of study subjects. For single arm studies, similar information was summarized. For studies that simply reported that they observed no adverse events, we created a simpler summary table listing only the information about the dosage, study size, and duration.