Criteria for Inclusion/Exclusion of Studies in the Review

Inclusion and exclusion criteria are described below according to the PICOTSS (population, intervention/intake, comparison group, outcome, time, setting, and study design) framework (Table 1). The criteria are based on the 2005 Institute of Medicine (IOM) report and on discussions with and recommendations of federal sponsors and the TEP for the current review. Studies that were considered for addressing Key Questions intended to assess the effect of interventions on the outcomes of interest (KQ 1, 3, 5, and 7) were limited to randomized controlled trials (RCTs) and controlled clinical trials (CCTs). Both parallel and crossover trials were included; however, based on concerns about possible carryover effects,²³ crossover trials that did not incorporate a minimum 2-week washout phase between treatment phases and did not explicitly describe the procedure used to ensure lack of carryover were excluded. If an article did not mention washout period duration or an effort to assess the potential for carryover, we searched for a separately published study protocol. If washout or a method to assess carryover were not mentioned in the protocol, the study was excluded.

Table 1

PICOTSS.

Studies that were considered in addressing Key Questions pertaining to the association between sodium and/or potassium intake and health effects included both prospective observational studies and multivariate analyses of results of RCTs in which randomization was not maintained. Included observational studies were limited to those studies that measured and quantified intake of sodium and/or potassium with valid indicators. Valid assessment measures were selected together with input from the Technical Expert Panel (TEP) and content expert and are described below in the section on assessment of risk of bias (RoB).

The Key Questions pertaining to associations excluded studies that exclusively followed participants with preexisting disease specific to the clinical outcome of interest. To use valid samples to determine associations, the cohort would need to include participants with and without the condition of interest at followup. Because the pool of association studies included observational studies where the exposure to a specific dietary strategy was self-selected and compared groups might differ in more characteristics then simply dietary sodium or potassium intake, eligible studies were limited to those reporting baseline data for the outcomes of interest.

The intervention or followup durations required for study inclusion (e.g., two years for studies on kidney disease) were determined by the federal sponsors, the TEP, and other clinical experts to ensure we included only studies with sufficient followup durations to detect the incident outcome of interest.

Other Exclusions Applying to All Key Questions

Only full-text peer-reviewed English-language publications were included. These decisions were made to ensure that sufficient study detail was provided and accessible to assess study quality fairly.

Searching for the Evidence

This section describes the literature search strategies, and screening protocols used.

Data Abstraction and Data Management

A detailed and standardized web-based data extraction form was used to record study-level information (see Protocol²⁴ for list of study-level variables) and RoB assessments for all studies that met inclusion criteria (Appendix B). The form was pilot-tested and refined within the review team. Data were extracted by one reviewer and checked by a second, senior systematic reviewer to ensure accuracy.

A number of studies had study-level details and/or outcomes reported in more than one publication. For those articles, abstractors ensured that the records were linked so that the correct study level data (e.g., baseline conditions for subgroups) were matched with outcome data and so that data were not abstracted in duplicate.

Outcome data, including confounders and effect modifiers, were abstracted into Excel spreadsheets and prepared for analysis by two members of the research team (one member extracted data from trials and one extracted data from observational studies) and were reviewed for accuracy by one of the PIs and the biostatistician. Data from studies that met inclusion criteria that were included in the 2005 DRI report or other systematic reviews were re-extracted.

All included studies are described in evidence tables (Appendix C). At the end of the project, all data will be uploaded to customized forms in Systematic Review Data Repository (SRDR) online system (http://srdr.ahrq.gov) for full public access.

Assessment of Methodological Risk of Bias of Individual Studies

We assessed the methodological RoB of each original study included in the review, based on predefined criteria.

We implemented the Cochrane Risk of Bias tool to assess RoB of RCTs, with criteria modified to cover concerns in the types of nutrition trials considered for this review. These modifications included considering bias that could arise if participants in parallel randomized controlled trials (RCTs) were not matched for (or were not at least similar regarding) body mass index (BMI), sodium excretion, age, gender, race/ethnicity, and hypertensive status; sodium intake assessment; adherence/compliance; absence of the outcome of interest at baseline; and use of appropriate statistical methods for assessing crossover trial outcomes (see Appendix E).

To assess RoB among observational studies, we used questions from the Newcastle Ottawa tool that are relevant for prospective studies (see Appendix E).^{25, 26} The RoB from the method used to assess sodium and potassium intakes was determined according to criteria described in Appendix E.²⁰ Other items assessed included similarity at baseline across treatment groups or quantiles regarding age, BMI, ethnicity, hypertensive status, and urinary sodium excretion.

An overall RoB was determined for each RCT by tabulating the numbers of individual “low,” “high,” “moderate,” and “unclear” scores. RCTs earned a low overall risk-of-bias rating if their total “low” scores were 8 or higher (out of 11) and their “high” scores were 1 or fewer and did not include intake assessment method; overall moderate ratings included 5 to 7 “low” scores and 2 or fewer “high” scores; overall high ratings included fewer than 4 “low” scores or more than 2 “high” scores; and overall unclear scores included 5 or more “unclear” ratings.

An overall RoB was determined for each observational study by giving the RoB of the method used to assess sodium or potassium intake the most weight and adjusting the grade down by tabulating the numbers of other individual “high,” “moderate” or “unclear” risk-of-bias domains. Observational studies earned a low overall risk-of-bias rating if the RoB of the method used to assess sodium or potassium intake was rated “low” and all other individual risk-of-bias domains were rated “low.” Observational studies earned a high overall risk-of-bias rating if their RoB of the method used to assess sodium or potassium intake was rated “high” or if the RoB of the method used to assess sodium or potassium intake was rated “moderate” and more than one other individual risk-of-bias domains were rated “high” or unclear.” The RoB of the method used to assess sodium or potassium intake was rated separately. As such, an observational study could receive different overall ratings for sodium- and potassium-outcome pairs.

One reviewer assessed the methodological RoB for all included studies and one other reviewer confirmed or refuted the RoB assessments. Disagreements were reconciled among the systematic review team and resolved via group consensus. When determining the overall strength of evidence, we considered any quality issues pertinent to the specific outcomes of interest.

Original studies whose references were reference mined from existing systematic reviews were screened, assessed for risk-of-bias, and data abstracted along with studies identified in literature searches.

Data Synthesis/Analysis

All included studies are presented in evidence tables (Appendix C and D). Continuous outcomes are reported as mean differences (MD), dichotomous incidence outcomes are reported as relative risks (RR), together with the 95% confidence interval (CI). Unless otherwise noted, the continuous outcomes are always mean differences in BP, expressed as mm Hg

In describing interventions and reporting the findings of pooled analyses, we did not attempt to define “reduced dietary sodium,” “low sodium,” or “increased potassium,” as definitions and target goals differed across studies, and some provided no definition or goal. However, for each sodium reduction RCT included in a pooled analysis, we report sodium or potassium intakes in terms of achieved 24-hour sodium excretion in the figures that accompany the text, and we report the weighted mean differences in sodium intakes in figure legends. For studies aimed at increasing potassium, we reported the levels of potassium provided and/or the 24-hour potassium excretion when reported.

Random effects meta-analyses using the Hartung-Knapp-Sidik-Jonkman method were conducted on RCTs of similar populations or subpopulations^27–30 (based on baseline comorbidities and nutrient status), implementation of similar interventions or use of similar intake measures, and use of compatible outcome measures. Each study is weighted by the inverse of its variance. In a random effects model, the variance includes the within-study variance along with the between-study variance. Studies including patients with pre-existing conditions specific to the clinical outcome of interest were excluded from analyses for the respective outcome of interest in this review, unless they report subgroup data where patients with pre-existing conditions were excluded. The findings are presented in forest plots.

To assess the possible effect of study quality on findings, we conducted sensitivity analyses for all meta-analyses of RCTs that had included studies with high or unclear RoB, omitting those studies. The findings are described along with the findings for the full analyses, and the resulting forest plots and lists of omitted studies appear in Appendix I.

To assess the possible effect of achieved sodium excretion by individual interventions, we did not conduct sensitivity analysis, but arranged the studies in each forest plot in the order of decreasing difference in sodium excretion between the intervention and control groups at followup. For selected comparisons, we describe the relationships between achieved sodium status and effect sizes.

Meta-regressions were conducted to assess whether other minerals affected outcomes of interest, if sufficient numbers of studies assessed these effects (KQ1a, 3a, 5a, 7a) and to compare differences between subgroups. Subgroup analyses were conducted when sufficient data were available to answer the subquestions on subpopulations of interest, i.e., sex, race/ethnicity, DRI age group(s) (1-3 y, 4-8y, 9-13y, 14-18y, 19-30y, 31-50y, 51-70y, and ≥71 y), reproductive status (pregnant and lactating women), as well as hypertensive status, diabetes, obesity (i.e., BMI ≥30), and renal health status for individual Key Questions.

The data for subgroups are reported in separate evidence tables (Appendix D).

Statistical heterogeneity was assessed and expressed as the I² statistic and considered in interpreting and weighing the results of meta-analyses.

If data from observational studies were sufficient (3 or more studies using 24-hour urinary excretion measures for each outcome), we would have performed both linear and non-linear dose-response meta-regressions to examine the associations between dietary intake levels and the risks of clinical outcomes using a two-stage hierarchical regression model.^{31, 32} It is important to note that this dose-response meta-regression model requires categorical exposure data (at least three exposure categories, including the reference category, within each study). No dose-response meta-regression could be performed for this report, due to lack of sufficient quantitative data. Thus, standard random effects meta-analysis was performed to pool reported linear trend estimates from the adjusted model assessing the association between 24-hour urinary excretion levels (continuous measure) and all-cause mortality outcomes (the only clinical outcome with sufficient data for pooling.

Summary of findings tables organized by Key Question, interventions or intakes, and key outcomes summarize the available evidence.

Grading the Strength of Evidence (SoE) for Major Comparisons and Outcomes

The project leaders assessed the strength of evidence (SoE) for the key outcomes listed in Table 2, based on guidance provided in the AHRQ Methods Guide. These outcomes were also used to answer the subquestions.

Table 2

Outcomes for determination of strength of evidence.

The SoE approach we used assesses the body of evidence for each conclusion based on five dimensions: study limitations (the RoB of the individual studies and the study designs), consistency (the degree to which included studies find the same direction of effect, within study designs), directness (for this report, we used directness to mean two things: whether the outcome in question is intermediary or clinical, but mainly, whether the assessment of moderating factors was based on a direct or indirect comparison, for example men compared with women), precision (the degree of certainty surrounding an effect estimate), and reporting bias (the likelihood that some findings were omitted from publication).

Four strength-of-evidence ratings were used—high, moderate, low, or insufficient—as defined below (Table 3). Bodies of evidence based entirely on pooled RCTs are considered to have a high strength of evidence, which can be down-graded for major concerns in each of the domains (study limitations, indirectness, inconsistency, imprecision, or suspected reporting bias). For example, a high strength of evidence conclusion would be based on a pooled analysis of (e.g.,) five or more RCTs sufficiently powered to assess the outcome of interest and with overall low RoB, with consistent findings across studies, relatively tight confidence intervals, and assessing a direct comparison. If overall RoB was high, if results were inconsistent, if confidence intervals were wide compared with the effect size (for mean differences), if the effect size was of borderline significance, or if the comparison of interest was indirect, we would downgrade one level for any one of those factors to moderate RoB or to low RoB for two or more of those factors. If the number of studies was insufficient to allow pooling or if only three or four small (particularly underpowered or inconsistent) studies could be included, we would downgrade to low strength of evidence. An insufficient strength of evidence was reserved for questions for which no more than two inconsistent RCTs were identified that addressed the question.

Table 3

Definitions of the levels of strength of evidence.

We used a similar approach for rating the strength of evidence based on observational studies (which were used to answer association questions), with several modifications detailed as follows. Bodies of evidence based on more than two large, population-based prospective cohort studies are considered to have a high strength of evidence, which can be down-graded for major concerns in each of the domains (study limitations, indirectness, inconsistency, imprecision, or suspected reporting bias). For each outcome, observational studies were first synthesized separately for each type of intake measurement method (i.e., 24-hour urinary excretions, estimated 24-hour urinary excretions, and self-report dietary assessment methods), and then synthesized across different types of intake measurement methods at the strength-of-evidence rating level. Almost all observational studies were synthesized qualitatively (within each type of intake measurement method). When assessing consistency or inconsistency across studies, the ranges of intake were taken into account, given that nutrient-outcome relationships may vary according to the ranges of intake. Thus, the consistency or inconsistency across studies can be assessed only within the same ranges of intakes. Overlapping in study populations was carefully considered in the qualitative synthesis to avoid double counting data from the same study cohort. Multiple publications from the same study cohort were retained in the review if they differed in study characteristics, outcome definitions, followup durations or statistical analyses.

For this review, we did not assess strengths of a body of evidence that included both RCTs and observational studies. However, if the RCT evidence is robust, observational studies may not contribute to strengthening the evidence unless they are high quality studies with large, precise effect sizes. Similarly, because of challenges in accounting for confounding, a body of evidence comprising only observational studies usually can provide only a low strength of evidence unless the studies demonstrate a very large effect, a strong dose-response association, or the observed effect cannot be accounted for by uncontrolled confounding.

Assessing Applicability

Applicability was assessed at the level of the total body of evidence for each conclusion. We considered the similarity of the population to the North American population in terms of mean baseline intakes/status of sodium and potassium, weight status, and baseline comorbidities, as well as age.

Peer Review and Public Commentary

Experts in the fields of nutrition, epidemiology and statistics, and medicine and individuals representing stakeholder and user communities were invited to provide external peer review of this draft systematic review; AHRQ and an associate editor provided comments. The draft report was posted on the AHRQ website for 4 weeks to elicit public comment. We addressed all reviewer comments, revised the text as appropriate, and will document everything in a disposition of comments report that will be made available 3 months after the Agency posts the final systematic review on the Effective Health Care Web site.