NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Morton SC, Adams JL, Suttorp MJ, et al. Meta-regression Approaches: What, Why, When, and How? Rockville (MD): Agency for Healthcare Research and Quality (US); 2004 Mar. (Technical Reviews, No. 8.)

Cover of Meta-regression Approaches

Meta-regression Approaches: What, Why, When, and How?

Show details

2Methodology

Systematic Review to Identify Meta-regression Publications

We searched the following library databases:

  • MEDLINE® 1966 - March, 2001
  • HealthSTAR 1975 - March, 2001
  • EMBASE 1974 - March, 2001
  • MANTIS 1880 - 2000
  • SciSearch® 1990 - March, 2001
  • Social SciSearch® 1974 - March, 2001
  • Allied and Complementary Medicine 1985 - 2000

The search terms were “metaregress-” or “meta” within two words of “regress-” (the latter also picks up the hyphenated form of the term, i.e., “meta-regression”). We used the same terms to search the Current Index to Statistics (1974 -1999). We also searched the Methodology Register of the Cochrane Library (version 1, 2001) using the keywords “meta-regression,” “metaregression,” or “regression.” We supplemented these searches with articles from the Southern California Evidence-Based Practice Center's methodological article database that contains over 500 articles, and canvassed experts, including our expert panel and draft report referees (described below), for additional references. We also searched the reference lists in all relevant articles for additional publications.

A single reviewer (Morton) reviewed all title lists for relevance. The full text for all relevant articles was obtained.

Common Statistical Notation Objective

Given the variety of meta-regression approaches available, our first analytic objective was to propose a common statistical framework using the knowledge gained from the articles found via our systematic review, in which all meta-regression models could be expressed.

Simulation Approach

We decided to implement a simulation study to compare the different meta-regression modeling approaches. Simulation allows us to set up a scenario (the “true” model), simulate data from that model, estimate parameters using various meta-regression models, and then compare the estimated parameters of each model with the true model (bias properties). We defer an evaluation of coverage for future research.

Our first question was: What methods work best under what circumstances? Sub-questions in this domain were:

  • What if the treatment effect depends on disease severity?
  • What if the studies have a wide range of population risks (e.g. control group mortality rates)?
  • What if there are relatively few studies?

Our second question was: What methods are most sensitive to assumptions? Sub-questions in this domain were:

  • Do random effects models “protect” against omitted variables?
  • If we are uncertain of the factors that modify treatment effects, what is the “safest” method to use?

Our approach was to hypothesize a person-level model, and generate data according to that model. We used this approach as medical intuition applies at the patient level, and treatment is applied at the patient level. We then aggregated the person-level data to the study level. This approach capitalizes on the data aggregation literature.19 Aggregation in this manner may allow us to work out aggregation bias properties in future research.

Panel Methodology

We convened a one-day meeting of nine nationally-recognized experts on heterogeneity, and meta-regression. Prior to the meeting, the experts were sent a meeting agenda, goals and objectives including key questions, a document containing our common notation, a discussion of our preliminary simulation, and our preliminary bibliography. We asked the experts to suggest additional meta-regression references. The list of experts and items from the meeting are shown in Appendix A.

During the meeting, four of the experts presented half-hour talks. The topics chosen spanned the different types of meta-regression approaches available:

  • Meta-analysis of multi-treatment studies20 (presented by Dr. Vic Hasselblad)
  • Control rate meta-regression models15, 21(presented by Dr. Chris Schmid)
  • Bayesian meta-analysis17 (presented by Dr. Thomas Louis)
  • Methodological challenges in meta-regression10, 11 (presented by Dr. Jesse Berlin)

The common notation and the preliminary simulation results were discussed in detail. During the last part of the meeting, the attendees broke into smaller groups to discuss the three topics of heterogeneity; meta-regression; and the simulation. Each group then reported on their group discussion to the entire panel. The experts reached agreement on the parameters needed to complete the simulation, and additional analyses to conduct. In addition, the experts provided advice on the second methodological topic the Southern California Evidence-Based Practice Center should address. Quality assessment for nonrandomized studies had been proposed. The meeting was audio-taped and transcribed to assist in the preparation of this report.

Views

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...