This glossary defines terms as they are used in reports produced by the Drug Effectiveness Review Project. Some definitions may vary slightly from other published definitions.

A

Adherence

Following the course of treatment proscribed by a study protocol.

Adverse effect

An adverse event for which the causal relation between the intervention and the event is at least a reasonable possibility.

Adverse event

An adverse outcome that occurs during or after the use of a drug or other intervention but is not necessarily caused by it.

Active-control trial

A trial comparing a drug in a particular class or group with a drug outside of that class or group.

Allocation concealment

The process by which the person determining randomization is blinded to a study participant’s group allocation.

B

Before-after study

A type nonrandomized study where data are collected before and after patients receive an intervention. Before-after studies can have a single arm or can include a control group.

Bias

A systematic error or deviation in results or inferences from the truth. Several types of bias can appear in published trials, including selection bias, performance bias, detection bias, and reporting bias.

Blinding

The process of preventing those involved in a trial from knowing to which comparison group a particular participant belongs. Trials are frequently referred to as “double-blind” without further describing if this refers to patients, caregivers, investigators, or other study staff.

C

Case series

A study reporting observations on a series of patients receiving the same intervention with no control group.

Case study

A study reporting observations on a single patient.

Case-control study

A study that compares people with a specific disease or outcome of interest (cases) to people from the same population without that disease or outcome (controls).

Clinically significant

A result that is large enough to affect a patient’s disease state in a manner that is noticeable to the patient and/or a caregiver.

Cohort study

An observational study in which a defined group of people (the cohort) is followed over time and compared with a group of people who were exposed or not exposed to a particular intervention or other factor of interest. A prospective cohort study assembles participants and follows them into the future. A retrospective cohort study identifies subjects from past records and follows them from the time of those records to the present.

Confidence interval

The range of values calculated from the data such that there is a level of confidence, or certainty, that it contains the true value. The 95% confidence interval is generally used in Drug Effectiveness Review Project reports. If the report is hypothetically repeated on a collection of 100 random samples of studies, the 100 resulting 95% confidence intervals will include the true population value 95% of the time.

Confounder

A factor that is associated with both an intervention and an outcome of interest.

Controlled clinical trial

A clinical trial that includes a control group but no or inadequate methods of randomization.

Convenience sample

A group of individuals being studied because they are conveniently accessible in some way. Convenience samples may or may not be representative of a population that would normally be receiving an intervention.

Crossover trial

A type of clinical trial comparing two or more interventions in which the participants, upon completion of the course of one treatment, are switched to another.

D

Direct analysis

The practice of using data from head-to-head trials to draw conclusions about the comparative effectiveness of drugs within a class or group. Results of direct analysis are the preferred source of data in Drug Effectiveness Review Project reports.

Dose-response relationship

The relationship between the quantity of treatment given and its effect on outcome. In meta-analysis, dose-response relationships can be investigated using meta-regression.

Double-blind

The process of preventing those involved in a trial from knowing to which comparison group a particular participant belongs. While double-blind is a frequently used term in trials, its meaning can vary to include blinding of patients, caregivers, investigators, or other study staff.

Double-dummy

The use of two placebos in a trial that match the active interventions when they vary in appearance or method of administration (for example, when an oral agent is compared with an injectable agent).

E

Effectiveness

The extent to which a specific intervention used under ordinary circumstances does what it is intended to do.

Effectiveness outcomes

Outcomes that are generally important to patients and caregivers, such as quality of life, responder rates, number and length of hospitalizations, and ability to work. Data on effectiveness outcomes usually comes from longer-term studies of a “real-world” population.

Efficacy

The extent to which an intervention produces a beneficial result under ideal conditions in a selected and controlled population.

Estimate of effect

The observed relationship between an intervention and an outcome. Estimate of effect can be expressed in a number of ways, including number needed to treat, odds ratio, risk difference, and risk ratio.

Equivalence level

The amount that an outcome from two treatments can differ but still be considered equivalent, as in an equivalence trial, or the amount that an outcome from treatment A can be worse than that of treatment B but still be considered noninferior, as in a noninferiority trial.

Equivalence trial

A trial designed to determine whether the response to two or more treatments differs by an amount that is clinically unimportant. This lack of clinical importance is usually demonstrated by showing that the true treatment difference is likely to lie between a lower and an upper equivalence level of clinically acceptable differences.

External validity

The extent to which reported results are generalizable to a relevant population.

F

Fixed-effect model

A model that calculates a pooled estimate using the assumption that all observed variation between studies is due to chance. Studies are assumed to be measuring the same overall effect. An alternative model is the random-effects model.

Forest plot

A graphical representation of the individual results of each study included in a meta-analysis and the combined result of the meta-analysis. The plot allows viewers to see the heterogeneity among the results of the studies. The results of individual studies are shown as squares centered on each study’s point estimate. A horizontal line runs through each square to show each study’s confidence interval—usually, but not always, a 95% confidence interval. The overall estimate from the meta-analysis and its confidence interval are represented as a diamond. The center of the diamond is at the pooled point estimate, and its horizontal tips show the confidence interval.

Funnel plot

A graphical display of some measure of study precision plotted against effect size that can be used to investigate whether there is a link between study size and treatment effect.

G

Generalizability

See External Validity.

H

Hazard ratio

The increased risk with which one group is likely to experience an outcome of interest. It is similar to a risk ratio. For example, if the hazard ratio for death for a treatment is 0.5, then treated patients are likely to die at half the rate of untreated patients.

Head-to-head trial

A trial that directly compares one drug in a particular class or group with another in the same class or group.

Heterogeneity

The variation in, or diversity of, participants, interventions, and measurement of outcomes across a set of studies.

I

I²

A measure of statistical heterogeneity of the estimates of effect from studies. Values range from 0% to 100%. Large values of I² suggest heterogeneity. I² is the proportion of total variability across studies that is due to heterogeneity and not chance. It is calculated as (Q−(n−1))/Q, where n is the number of studies.

Indirect analysis

The practice of using data from trials comparing one drug in a particular class or group with another drug outside of that class or group or with placebo and attempting to draw conclusions about the comparative effectiveness of drugs within a class or group based on those data. For example, direct comparisons between drugs A and B and between drugs B and C can be used to make an indirect comparison between drugs A and C.

Intention to treat

The use of data from a randomized controlled trial in which data from all randomized patients are accounted for in the final results. Trials often incorrectly report results as being based on intention to treat despite the fact that some patients are excluded from the analysis.

Internal validity

The extent to which the design and conduct of a study are likely to have prevented bias. Generally, the higher the internal validity, the better the quality of the study publication.

Inter-rater reliability

The degree of stability exhibited when a measurement is repeated under identical conditions by different raters.

Intermediate outcome

An outcome not of direct practical importance but believed to reflect outcomes that are important. For example, blood pressure is not directly important to patients but it is often used as an outcome in clinical trials because it is a risk factor for stroke.

L

Logistic regression

A form of regression analysis that models an individual's odds of disease or some other outcome as a function of a risk factor or intervention.

M

Mean difference

A method used to combine measures on continuous scales (such as weight) where the mean, standard deviation, and sample size are known for each group.

Meta-analysis

The use of statistical techniques in a systematic review to integrate the results of included studies. Although the terms are sometimes used interchangeably, meta-analysis is not synonymous with systematic review. However, systematic reviews often include meta-analyses.

Meta-regression

A technique used to explore the relationship between study characteristics (for example, baseline risk, concealment of allocation, timing of the intervention) and study results (the magnitude of effect observed in each study) in a systematic review.

Multivariate analysis

Measuring the impact of more than one variable at a time while analyzing a set of data.

N

N-of-1 trial

A randomized trial in an individual to determine the optimum treatment for that individual.

Noninferiority trial

A trial designed to determine whether the effect of a new treatment is not worse than a standard treatment by more than a prespecified amount. A one-sided version of an equivalence trial.

Nonrandomized study

Any study estimating the effectiveness (harm or benefit) of an intervention that does not use randomization to allocate patients to comparison groups. There are many types of nonrandomized studies, including cohort studies, case-control studies, and before-after studies.

Null hypothesis

The statistical hypothesis that one variable (for example, treatment to which a participant was allocated) has no association with another variable or set of variables.

Number needed to treat

An estimate of how many persons need to receive a treatment before 1 person would experience a beneficial outcome.

O

Observational study

A type of nonrandomized study in which the investigators do not seek to intervene, instead simply observing the course of events.

Odds ratio

The ratio of the odds of an event in one group to the odds of an event in another group. An odds ratio of 1.0 indicates no difference between comparison groups. For undesirable outcomes an odds ratio that is <1.0 indicates that the intervention was effective in reducing the risk of that outcome.

One-tailed test (one-sided test)

A hypothesis test in which the values that reject the null hypothesis are located entirely in one tail of the probability distribution. For example, testing whether one treatment is better than another (rather than testing whether one treatment is either better or worse than another).

Open-label trial

A clinical trial in which the investigator and participant are aware which intervention is being used for which participant (that is, not blinded). Random allocation may or may not be used in open-label trials.

P

Per protocol

The subset of participants from a randomized controlled trial who complied with the protocol sufficiently to ensure that their data would be likely to exhibit the effect of treatment. Per protocol analyses are sometimes misidentified in published trials as intention-to-treat analyses.

Point estimate

An estimate of what is true for a population based on results (for example, mean, weighted mean difference, odds ratio, risk ratio, or risk difference) obtained in a sample (a study or a meta-analysis) of that population.

Pooling

The practice of combining data from several studies to draw conclusions about treatment effects.

Power

The probability that a trial will detect statistically significant differences among intervention effects. Studies with small sample sizes can frequently be underpowered to detect difference.

Precision

The likelihood of random errors in the results of a study, meta-analysis, or measurement. The greater the precision, the less the random error. Confidence intervals around the estimate of effect are one way of expressing precision, with a narrower confidence interval meaning more precision.

Prospective study

A study in which participants are identified according to current risk status or exposure and followed forward through time to observe outcome.

Publication bias

A bias caused by only a subset of the relevant data being available. The publication of research can depend on the nature and direction of the study results. Studies in which an intervention is not found to be effective are sometimes not published. Because of this, systematic reviews that fail to include unpublished studies may overestimate the true effect of an intervention. In addition, a published report might present a biased set of results (for example, only outcomes or subgroups for which a statistically significant difference was found).

P value

The probability (ranging from zero to one) that the results observed in a study could have occurred by chance if the null hypothesis was true. A P value of ≤ 0.05 is often used as a threshold to indicate statistical significance.

Q

Q-statistic

A measure of statistical heterogeneity of the estimates of effect from studies. Large values of Q suggest heterogeneity. It is calculated as the weighted sum of the squared difference of each estimate from the mean estimate.

R

Random-effects model

A statistical model in which both within-study sampling error (variance) and between-studies variation are included in the assessment of the uncertainty (confidence interval) of the results of a meta-analysis. When there is heterogeneity among the results of the included studies beyond chance, random-effects models will give wider confidence intervals than fixed-effect models.

Randomization

The process by which study participants are allocated to treatment groups in a trial. Adequate (that is, unbiased) methods of randomization include computer generated schedules and random-numbers tables.

Randomized controlled trial

A trial in which two or more interventions are compared through random allocation of participants.

Regression analysis

A statistical modeling technique used to estimate or predict the influence of one or more independent variables on a dependent variable, for example, the effect of age, sex, or confounding disease on the effectiveness of an intervention.

Relative risk

The ratio of risks in two groups; same as a risk ratio.

Retrospective study

A study in which the outcomes have occurred prior to study entry.

Risk difference

The difference in size of risk between two groups.

Risk ratio

The ratio of risks in two groups. In intervention studies, it is the ratio of the risk in the intervention group to the risk in the control group. A risk ratio of 1 indicates no difference between comparison groups. For undesirable outcomes, a risk ratio that is <1 indicates that the intervention was effective in reducing the risk of that outcome.

S

Sensitivity analysis

An analysis used to determine how sensitive the results of a study or systematic review are to changes in how it was done. Sensitivity analyses are used to assess how robust the results are to uncertain decisions or assumptions about the data and the methods that were used.

Standard deviation (SD)

A measure of the spread or dispersion of a set of observations, calculated as the average difference from the mean value in the sample.

Standard error (SE)

A measure of the variation in the sample statistic over all possible samples of the same size. The standard error decreases as the sample size increases.

Statistically significant

A result that is unlikely to have happened by chance.

Subgroup analysis

An analysis in which an intervention is evaluated in a defined subset of the participants in a trial, such as all females or adults older than 65 years.

Superiority trial

A trial designed to test whether one intervention is superior to another.

Systematic review

A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research and to collect and analyze data from the studies that are included in the review.

T

Tolerability

Unpleasant adverse effects of drugs that are usually transient and not clinically significant, although they can affect a person’s quality of life and willingness to continue a treatment.

Two-tailed test (two-sided test)

A hypothesis test in which the values that reject the null hypothesis are located in both tails of the probability distribution. For example, testing whether one treatment is different than another (rather than testing whether one treatment is better than another).

Type I error

A conclusion that there is evidence that a treatment works, when it actually does not work (false-positive).

Type II error

A conclusion that there is no evidence that a treatment works, when it actually does work (false-negative).

V

Validity

The degree to which a result (of a measurement or study) is likely to be true and free of bias (systematic errors).