Quality Review for observational trials (Originally in EXCEL)

ColumnCategoryQuestion
AREFID
BReviewer initial
CBackground/ContextIs the hypothesis/aim/objective of the study described?

Yes
No
DSample Definition and SelectionAre the inclusion/exclusion criteria clearly stated (does not require the reader to infer)? [Abstractor: use “Partially” if only some criteria are stated clearly.]

Yes
Partially
No
EDid the authors report conducting a power analysis or some other basis for determining the adequacy of study group sizes for the primary outcome(s) being abstracted?

Yes
No
FInterventions/ExposureWhat is the level of detail in describing the intervention or exposure?
Intensity, duration, frequency, setting and timing

Low (unclear, many details missing)
Medium (pretty clear, most details provided)
High (very clear, all required details provided)
GIs usual clinical care (sometimes called standard care) described?

Yes
No
NA (not an intervention study)
HContaminationDid researchers rule out any impact from an unintended intervention/exposure that might bias results, e.g., through multivariate analysis, stratification, or subgroup analysis?

Yes
No
NA (no unintended interventions reported)
ICould variation from the protocol have compromised the findings of study?

Yes (variation from protocol exists and could have compromised findings)
No (variation from protocol exists, but unlikely to have compromised findings)
Cannot determine (no variation from protocol reported)
NA (study does not require protocol, or no variation from protocol exists)
JBlindingWere the outcome assessors blinded to the intervention or exposure status of participants?

Yes
No
NA (not an intervention study)
KSoundness of informationAre interventions/exposures measured in a valid and reliable manner?

Objective (clinical reports, lab findings, previously validated measures)
Objective measure, not validated
Prospective documentation (including self-report in daily diaries)
Retrospective self-report (patient/participant response)
Not reported
LAre outcomes measured in a valid and reliable manner?

Objective (clinical reports, lab findings, previously validated measures)
Objective measure, not validated
Prospective documentation (including self-report in daily diaries)
Retrospective self-report (patient/participant response)
Not reported
MFollow-upIn cohort studies, do the analyses adjust for different lengths of follow-up of patients, or in case-control studies, is the time period between the intervention/exposure and outcome the same for cases and controls? [Abstractor: Where follow-up was the same for all study patients the answer is yes. If different lengths of follow-up were adjusted by, for example, survival analysis, the answer is yes. Studies where differences in follow-up are ignored should be answered NA.]

Yes
No
Cannot determine
NA (cross-sectional)
NIs the length of time following the intervention/exposure sufficient to support the conclusions of the study regarding outcomes?

Yes
No
NA (cross-sectional)
ODid attrition from any group exceed 20 percent (after allocation of treatment)?

Yes - how much?
No
Cannot determine
NA (cross sectional)
PDid attrition differ between groups by more than 15 percentage points (after allocation of treatment)?

Yes - how much?
No
Cannot determine
NA (cross sectional)
QAnalysis comparabilityAre baseline characteristics similar in exposed and comparison cohorts?

Yes
No
Cannot determine
NA (case series)
RDoes the analysis control for baseline differences?

Yes
No
Cannot determine
NA (no baseline differences reported)
SWere the important confounding and modifying variables taken into account in the design and analysis (e.g., through matching, stratification, or statistical adjustment)?

Yes
Partially
No
Cannot determine
TAnalysis OutcomeIs the analysis conducted on an intention-to-treat (ITT) basis, that is, the intervention allocation status rather than the actual intervention received?

Yes
No
UIs the impact of loss to follow-up (or differential loss to followup) assessed (e.g. through sensitivity analysis or other intention-to- treat adjustment methods?

Yes
No
Cannot determine
NA (cross-sectional or case-control selected on outcome)
VAre the statistical methods used to assess the primary outcomes appropriate to the data? [Abstractor: The statistical techniques used must be appropriate to the data. For example, non-parametric methods should be used for small sample sizes (N<30). If studies have not accounted for differences between the unit of allocation and the unit of analysis, (e.g., through mixed models or generalized estimating equations for analysis of individual covariates or through t-tests or weighted t-tests for cluster-level analysis) then the answer is no. If outcomes are rare and little or no statistical analysis has been conducted, answer yes if studies have accounted for alternative causes other than the intervention/exposure. For details on whether specific statistical tests are appropriate, go to http://bama​.ua.edu/~jleeper​/627/choosestat.html.4]

Yes
Partially
No
NA (not reported)
WFor cohort studies only, if the outcome has a greater than 10 percent prevalence, is the risk ratio and relative risk calculated directly (not using logistic regression)?

Yes
No
NA (not a cohort study)
XDoes the study report appropriate estimates of the random variability in the data for the main outcomes?4 [Abstractors: In non- normally distributed data the inter-quartile range of results should be reported. In normally distributed data the standard error, standard deviation or confidence intervals should be reported.]

Yes
No
YInterpretationAre conclusions supported by results with possible bias and limitations taken into consideration?

Yes
Partially
No
ZQualityGood
Fair
Poor

From: Appendix B, Abstract Forms

Cover of Outcomes of Community Health Worker Interventions
Outcomes of Community Health Worker Interventions.
Evidence Reports/Technology Assessments, No. 181.
Viswanathan M, Kraschnewski J, Nishikawa B, et al.

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.