This Technical Support Document focuses on heterogeneity in relative treatment effects. Heterogeneity indicates the presence of effect-modifiers. A distinction is usually made between true variability in treatment effects due to variation between patient populations or settings, and biases related to the way in which trials were conducted. Variability in relative treatment effects threatens the external validity of trial evidence, and limits the ability to generalise from the results, imperfections in trial conduct represent threats to internal validity. In either case it is emphasised that, although we continue to focus attention on evidence from trials, the study of effect-modifying covariates is in every way a form of observational study, because patients cannot be randomised to covariate values. This document provides guidance on methods for outlier detection, meta-regression and bias adjustment, in pair-wise meta-analysis, indirect comparisons and network meta-analysis, using illustrative examples.

Guidance is given on the implications of heterogeneity in cost-effectiveness analysis. We argue that the predictive distribution of a treatment effect in a “new” trial may, in many cases, be more relevant to decision making than the distribution of the mean effect. Investigators should consider the relative contribution of true variability and random variation due to biases, when considering their response to heterogeneity.

Where subgroup effects are suspected, it is suggested that a single analysis including an interaction term is superior to running separate analyses for each subgroup.

Three types of meta-regression models are discussed for use in network meta-analysis where trial-level effect-modifying covariates are present or suspected: (1) Separate unrelated interaction terms for each treatment; (2) Exchangeable and related interaction terms; (3) A single common interaction term. We argue that the single interaction term is the one most likely to be useful in a decision making context. Illustrative examples of Bayesian metaregression against a continuous covariate and meta-regression against “baseline” risk are provided and the results are interpreted. Annotated WinBUGS code is set out in an Appendix. Meta-regression with individual patient data is capable of estimating effect modifiers with far greater precision, because of the much greater spread of covariate values. Methods for combining IPD in some trials with aggregate data from other trials are explained.

Finally, four methods for bias adjustment are discussed: meta-regression; use of external priors to adjust for bias associated with markers of lower study quality; use of network synthesis to estimate and adjust for quality-related bias internally; and use of expert elicitation of priors for bias.