Table 1Types of selective outcome and analysis reporting, which may affect the direction and/or magnitude of the reported study findings

Selective Outcome ReportingSelective Analysis Reporting
Missing/changed outcomes:
  • Omission of an outcome that was prespecified or for which the clinical judgment of the review team strongly suggests should have been prespecified
  • Addition of an outcome that was not prespecified (excluding unintended or unanticipated harms outcomes)
  • Change from the protocol in a primary or secondary outcome
  • Failure to report prespecified subgroups
  • Reporting of a composite outcome without reporting of results for individual components, or reporting of composites of unconventional components
  • Use of a different outcome measurement tool or definition from that prespecified in the protocol without a reasonable justification
  • Incomplete specification of an outcome domain (e.g., ‘substance use’ versus ‘abstinence’ or ‘reduction in use’) and specific measurement (e.g., self-reported measures versus levels in biologic tissues) in the methods section of the publication or in other available sources
Incomplete reporting
  • Partial reporting of outcomes (in other words, information is not sufficient to add the study to a meta-analysis) for example: including an absolute or relative measure without either a confidence interval or a precise p value
  • Use of inexact p values (except p<0.01, which does not require more precision)
  • Narrative presentation of quantitative results (e.g., “significant” or “not significant”)
Changes to/in (planned), or selection from (multiple):
  • Data types, for example, dichotomous instead of continuous using favorable post hoc cut-offs
  • Effect measure specific metric or method of aggregation, for example, reporting of the more favorable of the change-from-baseline (change score) or the final value comparison for a continuous outcome when both were analyzed
  • Assumptions of data distribution or estimate adjustments without reasonable justification
  • Time points for analysis
  • Post hoc subgroup analyses
  • Selectively reporting the first period results in crossover trials

From: Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions: AHRQ and the Effective Health Care Program

Cover of Methods Guide for Effectiveness and Comparative Effectiveness Reviews
Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet].

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.