StudyDesignPrimary Reasons for Poor-Quality Rating
Arozullah et al., 20062Cross-sectionalHigh potential for selection biases. A convenience sample with a low participation rate was used.
Bennett et al., 20063Retrospective cohortHigh potential for selection and confounding biases. A convenience sample with no power calculation was used and there was no controlling for confounding in the analysis.
Bickmore et al., 20094RCTHigh potential for selection and measurement bias. The process of randomization was inadequate, there was no allocation concealment, groups were not comparable at baseline, and there was inadequate controlling for confounding in the analysis.
Brock et al., 20075Uncontrolled experimental study (pre/post test)This study received a fair rating for immediate outcomes but a poor rating for follow-up outcomes. There was a high risk for selection and confounding bias at followup due to high likelihood that the groups were no longer comparable and inadequate controlling for potential confounders in the analysis.
Campbell et al., 20076Cross-sectionalHigh potential for confounding and selection biases. A convenience sample was used.
Carbone et al., 20067Cross-sectionalHigh potential for measurement bias. Outcome measures were poorly described and could not be considered valid and reliable.
Clarke et al., 20058Cross-sectionalHigh potential for selection bias. Reporting of measures and statistical methods was inadequate. Important potential confounders were not considered.
Conwell et al., 20039Cross-sectionalHigh risk for confounding bias: race, socioeconomic status, parental smoking status, behavioral status, or any other potential confounder, could be responsible for association between WRAT score and smoking status.
Cordasco et al., 200910RCTFalse inclusions and attrition-introduced selection bias and residual confounding that was not controlled for in analysis.
DeWalt et al., 200711Cross-sectionalHigh potential for selection and confounding biases. A convenience sample with no power calculation was used and there was no controlling for confounding in the analysis.
DeWalt et al., 200912Uncontrolled experimental study (pre/post test)High risk of measurement bias due to social desirability. There was also inadequate controlling for confounding in the analysis.
DeWalt et al., 200413Uncontrolled experimental study (pre/post test)High risk of measurement and confounding bias. The lack of a control group carries a significant risk that any improvement in clinical symptoms was due to a Hawthorne effect or the use of cointerventions.
Donelle et al., 200814Cross-sectionalLiteracy/numeracy groups very likely to be different and only age/gender controlled for as potential confounders. Furthermore, comprehension questions were nonvalidated and not clearly appropriate.
Drainoni et al., 200815Cross-sectionalHigh potential for measurement, selection, and confounding biases. Outcome measures were poorly described and could not be considered valid and reliable. A convenience sample with no power calculation was used and there was no controlling for confounding in the analysis.
Endres et al., 200416Cross-sectionalHigh potential for selection and confounding biases. A small convenience sample was used and there was no controlling for important potential confounders in the analysis.
Garcia-Retamero and Galesic, 200917Factorial RCTThis study received a fair rating for main effect but a poor rating for subgroup analyses, with no presentation of baseline characteristics by group. There was no control of potential confounders if participants exited, making selection and confounding major issues.
Garcia-Retamero and Galesic, 201018RCTLack of adequate reporting about study, unclear what the study design is for between-group comparisons, unclear sample size and baseline numeracy/graphical literacy. No control for confounding in between-group analyses and subgroup analyses (although not clear whether needed for main group analyses).
Gazmararian et al., 201019Nonrandomized trialNonrandomized trial with no baseline differences and no control for confounding. Additionally, the author stated that the trial was underpowered, but it is not clear for what difference/outcomes.
Ginde et al., 200820Cross-sectionalHigh potential for measurement and confounding biases. Outcome measures were poorly described and could not be considered valid and reliable. There was no controlling for important potential confounders in the analysis.
Ives et al., 200621Prospective cohortHigh potential for confounding bias. Bivariate analysis was used with no controlling for important potential confounders in the analysis.
Jones et al., 200722Cross-sectionalHigh potential for measurement, selection, and confounding biases. Outcome measures were poorly described and could not be considered valid and reliable. A convenience sample with no power calculation was used and there was no controlling for confounding in the analysis.
Juzych et al., 200823Cross-sectionalHigh potential for confounding bias. Bivariate analysis was used with no controlling for important potential confounders in the analysis.
Kalichman et al., 200524Uncontrolled experimental study (pre/post test)High risk of measurement and confounding bias due to social desirability and inadequate controlling for confounding in the analysis.
Kandula et al., 200925Cross-sectional; prospective cohortHigh potential for measurement bias. Outcome measures were poorly described and could not be considered valid and reliable.
Kleinpeter, 200326Cross-sectionalHigh potential for selection and confounding biases. A small convenience sample was used and there was no controlling for important potential confounders in the analysis.
Lincoln et al., 200827Cross-sectionalHigh potential for selection biases A small convenience sample was used and participation rate was low.
Mbaezue et al., 201028Cross-sectionalHigh potential for measurement and selection bias. Descriptive data in tables do not add to the total sample. A portion of the sample population that did not check its glucose was omitted, causing the multivariate model to be misspecified.
Morrow et al., 200629Cross-sectionalHigh potential for selection and confounding bias. Health outcome measure poorly described.
Muir et al., 200630Retrospective cohortHigh potential for confounding bias. Bivariate analysis was used with no controlling for important potential confounders in the analysis.
Ntri et al., 200931Uncontrolled experimental study (pre/post test)High potential for confounding and selection biases. There was no controlling for potential confounders in the analysis and no accounting for those lost to followup. A small convenience sample was used.
Persell et al., 200732Cross-sectionalHigh potential for confounding biases. There was no controlling for important potential confounders in the analysis.
Roth et al., 200533Cross-sectionalHigh potential for selection and confounding biases. A small convenience sample was used and there was no controlling for important potential confounders in the analysis.
Rutherford et al., 200634Cross-sectionalHigh potential for measurement and confounding biases. Outcome measures were poorly described and could not be considered valid and reliable. There was inadequate controlling for important potential confounders in the analysis.
Sanders et al., 200735Retrospective cohortHigh potential for measurement bias. Outcome measures were poorly described and could not be considered valid and reliable.
Sarkar et al., 200636Cross-sectionalHigh potential for confounding biases. A convenience sample was used and there was inadequate controlling for important potential confounders in the analysis.
Sentell et al., 200337Cross-sectionalHigh potential for measurement and confounding biases. The outcome was measured by a single-item, self-reported survey question and there was inadequate controlling for important potential confounders in the analysis because only the bivariate analyses were relevant to the outcome of interest for this report.
Shieh et al., 200938Cross-sectionalHigh potential for confounding and measurement bias. Inadequate control for confounding and the outcome measure could not be considered valid and reliable.
van Servellen et al., 2003 & 200539,40RCTHigh potential for measurement and confounding biases. Inadequate reporting. Important potential confounders and multiple comparisons were not considered in the analysis and the analysis was within not between groups.
Waldrop-Valverde et al., 200841Cross-sectionalHigh potential for measurement and selection biases. The sample was divided into literacy/cognition groups so the independent effect of literacy on adherence could not be determined.
Wallace et al., 200842Cross-sectionalHigh potential for confounding bias. Bivariate analysis was used with no controlling for important potential confounders in the analysis.
Wolf et al., 200443Cross-sectionalHigh potential for measurement and confounding biases. Outcome measures were poorly described and could not be considered valid and reliable. There was inadequate controlling for important potential confounders in the analysis.
Wolf et al., 200744Cross-sectionalHigh potential for measurement and confounding biases. Outcome measures were poorly described and could not be considered valid and reliable. There was inadequate controlling for important potential confounders in the analysis.

RCT= Randomized controlled Trial

From: Appendix E, Characteristics of Studies with Poor Internal Validity

Cover of Health Literacy Interventions and Outcomes: An Updated Systematic Review
Health Literacy Interventions and Outcomes: An Updated Systematic Review.
Evidence Reports/Technology Assessments, No. 199.
Berkman ND, Sheridan SL, Donahue KE, et al.

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.