• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of biometLink to Publisher's site
Biometrika. Jun 2010; 97(2): 321–332.
Published online Apr 15, 2010. doi:  10.1093/biomet/asq006
PMCID: PMC3412575

On the relative efficiency of using summary statistics versus individual-level data in meta-analysis

Summary

Meta-analysis is widely used to synthesize the results of multiple studies. Although meta-analysis is traditionally carried out by combining the summary statistics of relevant studies, advances in technologies and communications have made it increasingly feasible to access the original data on individual participants. In the present paper, we investigate the relative efficiency of analyzing original data versus combining summary statistics. We show that, for all commonly used parametric and semiparametric models, there is no asymptotic efficiency gain by analyzing original data if the parameter of main interest has a common value across studies, the nuisance parameters have distinct values among studies, and the summary statistics are based on maximum likelihood. We also assess the relative efficiency of the two methods when the parameter of main interest has different values among studies or when there are common nuisance parameters across studies. We conduct simulation studies to confirm the theoretical results and provide empirical comparisons from a genetic association study.

Some key words: Cox regression, Evidence-based medicine, Genetic association, Individual patient data, Information matrix, Linear regression, Logistic regression, Maximum likelihood, Profile likelihood, Research synthesis

1. Introduction

Meta-analysis, the combination of results from a series of independent studies, is gaining popularity in many fields, including medicine, psychology, epidemiology, education, genetics and ecology. In particular, meta-analysis publications in medical research have grown enormously over the last three decades, due to greater emphasis on evidence-based medicine and the need for reliable summarization of the vast and expanding volume of clinical research (e.g. Sutton et al., 2000; Whitehead, 2002). Most of the recent discoveries on genetic variants influencing complex human diseases were made possible through meta-analysis of multiple studies (e.g. Lohmueller et al., 2003; Zeggini et al., 2008).

Traditionally, meta-analysis is carried out by combining the summary statistics of relevant studies, which are available in journal articles. With improving technologies and communications and increasing recognition of the benefits of meta-analysis, it is becoming more feasible to access the raw or original data on individual participants (e.g. Sutton et al., 2000). Indeed, meta-analysis of individual patient data is regarded as the gold standard in systematic reviews of randomized clinical trials (e.g. Chalmers et al., 1993). Recently, a number of networks and consortia have been created to share original data from genetic association studies (e.g. Kavvoural & Ioannidis, 2008; The Psychiatric GWAS Consortium Steering Committee, 2009). In general, obtaining original data is difficult, costly and time-consuming. A question naturally arises as to how much efficiency gain can be achieved by analyzing original or individual-level data over combining summary statistics.

A partial answer to this question was provided by Olkin & Sampson (1998), who showed that, in the case of comparing multiple treatments and a control with respect to a continuous outcome, the traditional meta-analysis based on estimated treatment contrasts is equivalent to the least-squares regression analysis of individual patient data if there are no study-by-treatment interactions and the error variances are constant across trials. Mathew & Nordström (1999) claimed that the equivalence holds even if the error variances are different across trials. There has been no theoretical investigation beyond this special setting. Empirically, meta-analysis using original data has been found to be generally similar but not identical to meta-analysis using summary statistics (e.g. Whitehead, 2002, Ch. 5).

In the present paper, we provide a systematic investigation into the relative efficiency of using summary statistics versus original data in fixed-effects meta-analysis, which assumes a common effect among studies. We prove that the two types of meta-analysis are asymptotically equivalent for all commonly used parametric and semiparametric models provided that the effect sizes are indeed the same for all studies, the nuisance parameters have different values across studies, and maximum likelihood estimation is used in the calculations of summary statistics and in the joint analysis of original data. We also investigate the relative efficiency of the two methods when the effect sizes are different among studies or when there are common nuisance parameters across studies. We illustrate the theoretical results with simulated and empirical data.

2. Theoretical results

2.1. Main results

Suppose that there are K independent studies, with nk participants for the kth study. The original data consist of (Yki, Xki) (k = 1, . . ., K ; i = 1, . . ., nk), where Yki is the response variable for the ith participant of the kth study, and Xki is the corresponding vector of explanatory variables. The response variable can be continuous or discrete, univariate or multivariate. Under fixed-effects models, the conditional density of Yki given Xki takes the form f (y, x; β, ηk), where β is a vector of parameters common to all K studies, and ηk is a vector of parameters specific to the kth study. A simple example is the linear regression model for the normal response variable:

Yki=αk+βTXki+ki(k=1,,K;i=1,,nk),

where [sm epsilon]ki is normal with mean zero and variance σk2 (Whitehead, 2002, § 5.2.1). In this case, f (y, x; β, ηk) = (2π σk2)−1/2 exp{−(yαkβTx)2/2 σk2}, and ηk = (αk, σk2). Additional examples are given in § 2.4. We wish to make inference about β.

Meta-analysis is usually performed on a scalar parameter. We allow vector-valued β for two reasons. First, there are important applications in which the effects of interest, such as treatment differences in a multi-arm clinical trial or co-dominant effects of a genetic variant, are truly multivariate. Second, if the nuisance parameters, e.g. intercepts or confounding effects, have the same values among the K studies, then performing meta-analysis jointly on the effects of interest and the common nuisance parameters can improve statistical efficiency, as will be discussed in § 2.2. Of course, our formulation includes scalar β as a special case.

Let [beta]k be the maximum likelihood estimator of β by maximizing the kth study likelihood:

Lk(β,ηk)=i=1nkf(Yki,Xki;β,ηk),

and let beta be the maximum likelihood estimator of β by maximizing the joint likelihood

L(β,η1,,ηK)=k=1KLk(β,ηk).

The profile likelihood functions for β based on Lk (β, ηk) and L(β, η1, . . ., ηK) are, respectively,

plk(β)=supηkLk(β,ηk)

and

pl(β)=supη1,,ηKL(β,η1,,ηK).

The corresponding observed profile information matrices are x2110k (β) = −[partial differential]2 log plk (β)/[partial differential]β2 and x2110(β) = −[partial differential]2 log pl(β)/[partial differential]β2. The maximizer of the profile likelihood is the same as the maximum likelihood estimator in that [beta]k = argmax plk (β) and beta = argmax pl(β). Write n = ∑k nk and assume that nk/nck [set membership] (0, 1) as n → ∞. Assume also that the regularity conditions for profile likelihood as stated in Murphy & van der Vaart (2000) hold. Then nk k1 ([beta]k) and nx2110−1(beta) are consistent estimators of the covariance matrices of nk1/2([beta]kβ) and n1/2(betaβ), respectively.

Remark 1. For survival data and other censored data, the likelihood needs to be modified. If Yki is right censored at ki, then we replace f (Yki, Xki; β, ηk) in the likelihood by S(ki, Xki; β, ηk), where S(y, x; β, ηk) = yf(u,x;β,ηk)du.

Remark 2. Our framework allows different likelihood functions among studies, and the statistical models are not necessarily regression models. In meta-analysis of diagnostic accuracy data, the likelihood for each study pertains to the multinomial distribution of a 2 × 2 contingency table, and β can be sensitivity or specificity or both.

In traditional meta-analysis, one collates summary statistics [beta]k and vâr([beta]k) = 1/x2110k ([beta]k) (k = 1, . . ., K) for a scalar parameter β. The well-known inverse-variance estimator of β is

β^=k=1Kβ^k/va^r(β^k)k=1K1/va^r(β^k),
(1)

and its variance is estimated by

va^r(β^)=1k=1K1/va^r(β^k).
(2)

To allow vector-valued β, we use a multivariate version of estimator (1):

β^={k=1Kk(β^k)}1k=1Kk(β^k)β^k,
(3)

whose covariance matrix is estimated by

va^r(β^)={k=1Kk(β^k)}1.
(4)

Clearly, (3) and (4) reduce to (1) and (2) if β is a scalar. If original data are available, one can estimate β by the maximum likelihood estimator beta, whose covariance matrix is estimated by vâr(beta) = x2110−1(beta). It is easy to verify that pl(β)=k=1Kplk(β), which implies that (β)=k=1Kk(β). Thus,

va^r(β˜)={k=1Kk(β˜)}1.
(5)

Equations (4) and (5) show that vâr([beta]) and vâr(beta) take the same form; the only difference is that x2110k (β) are evaluated at [beta]k in the former and at beta in the latter. Under standard regularity conditions, [beta]k (k = 1, . . ., K) and beta converge to β, and nk1 x2110k (β) (k = 1, . . ., K) converge to constant matrices. It follows that n1/2([beta]β) and n1/2(betaβ) have the same limiting normal distribution. Thus, using summary statistics has the same asymptotic efficiency as using original data.

2.2. Common nuisance parameters

According to the results of the last section, meta-analysis based on summary statistics has the same asymptotic efficiency as the maximum likelihood estimator of full data if the former analysis is performed jointly on all common parameters. It is generally difficult to obtain multivariate summary statistics, especially in retrospective meta-analysis of published results. Thus, it is important to determine the efficiency loss of meta-analysis based on the univariate summary statistics for the effect of main interest when there are other common effects, which are referred to as common nuisance parameters.

Suppose that β is a scalar parameter representing a common effect of main interest and that a subset of ηk, denoted by γ, is a vector of common nuisance parameters. Denote the profile information matrices for (β, γ) based on the kth study data and all the data as

k=(kββkβγkγβkγγ)and=(βββγγβγγ),

respectively. The variance of [beta]k is approximately (x2110kββx2110kβγ kγγ1x2110kγβ)−1, so the variance of [beta] is approximately {∑k (x2110kββx2110kβγ kγγ1x2110kγβ)}−1. The variance of beta is approximately (x2110ββx2110βγ γγ1 x2110γβ)−1. Thus, the relative efficiency of [beta] to beta is approximately

kkββkkβγkγγ1kγββββγγγ1γβ.

Because x2110 = ∑k x2110k, the relative efficiency can also be expressed as

kkββkkβγkγγ1kγβkkββ(kkβγ)(kkγγ)1(kkγβ).

It follows from Lemma 1 of Appendix A that

kkβγkγγ1kγβ(kkβγ)(kkγγ)1(kkγβ).

Thus, the relative efficiency is always less than or equal to 1. It also follows from Lemma 1 that the relative efficiency is 1 if and only if x2110kβγ kγγ1 = x2110lβγ lγγ1 (∀k ≠ l). Note that x2110kβγ kγγ1 ≈ − var([beta]k)−1 cov ([beta]k, [gamma with circumflex]k), where [beta]k and [gamma with circumflex]k are the maximum likelihood estimators of β and γ based on the kth study data. Thus, the relative efficiency is 1 if and only if var([beta]k)−1cov([beta]k, [gamma with circumflex]k) are the same among the K studies. Obviously, the latter condition is satisfied if cov([beta]k, [gamma with circumflex]k) = 0 (k = 1, . . ., K). The foregoing conclusions also hold for multivariate β.

2.3. Unequal effect sizes

The fixed-effects meta-analysis assumes that the effect sizes βk are the same across studies. This assumption does not affect the Type I error of hypothesis testing since all effect sizes are the same under the null hypothesis. However, it is of practical importance to determine the relative power of using summary statistics versus original data when the βk are unequal.

Write Uk (β) = [partial differential]log plk (β) and U(β) = [partial differential] log pl(β). By definition, Uk ([beta]k) = 0 (k = 1, . . ., K) and U(beta) = 0. Since U(β)=k=1KUk(β), we have k=1K{Uk(β^k)Uk(β˜)}=0. By the mean-value theorem, k=1Kk(βk*)(β^kβ˜)=0, where βk* lies between [beta]k and beta. In other words,

β˜={k=1Kk(βk*)}1k=1Kk(βk*)β^k.
(6)

Comparison of (6) to (3) reveals that beta is the same kind of weighted combination of [beta]k as [beta], with the weights x2110k (β) evaluated at βk* rather than [beta]k. As shown in § 2.1, the only difference between vâr(beta) and vâr([beta]) lies in the evaluation of x2110k (β) at beta versus [beta]k. Hence, meta-analysis based on summary statistics and meta-analysis of original data will have similar power provided that the x2110k (β) do not change their values drastically when β varies between [beta]k and beta. When the βk are unequal, the limits of [beta] and beta pertain to weighted combinations of βk, rather than a common parameter in a statistical model.

We consider the local alternatives H1n : βk(n) = β + O(n−1/2) (k = 1, . . ., K). Under H1n, the estimators [beta]k (k = 1, . . ., K) converge in probability to β. We show in Appendix B that beta also converges in probability to β under H1n. It follows that, for each k, the weights n−1 x2110k ([beta]k), n−1 x2110k (beta) and n−1 x2110k ( βk*) all converge in probability to the same constant matrix. Thus, meta-analysis based on summary statistics and meta-analysis of original data have the same asymptotic power against H1n.

For hypothesis testing, one can use score statistics in meta-analysis. For testing the null hypothesis H0 : β = β0, the score statistics based on summary statistics and original data are

{k=1KUk(β0)}T{k=1Kk(β0)}1{k=1KUk(β0)},

and UT(β0)x2110−1(β0)U (β0), respectively. The two statistics are numerically identical since U(β)=k=1KUk(β) and (β)=k=1Kk(β). This equivalence holds whether the true effect sizes are equal or not.

2.4. Special cases

In this section, we consider the three most common cases of meta-analysis: linear regression for continuous response, logistic regression for binary response and Cox regression for potentially censored survival time. We will pay particular attention to the form of the observed profile information matrices x2110k because, as shown in §§ 2.1 and 2.3, the only difference between [beta] and beta lies in the argument of the profile information matrix. We use X to denote the explanatory variables of main interest and Z to denote the unit component and possibly other explanatory variables or covariates. The numbers and types of covariates need not be the same among the K studies.

Example 1. Assume that the distribution of Yki conditional on Xki and Zki is normal with mean βTXki + γkT Zki and variance σk2. The observed profile information matrix for β the kth study data is k(σk2)=Dk/σk2, where

Dk=i=1nkXki2i=1nkXkiZkiT(i=1nkZki2)1i=1nkZkiXkiT,

and a[multiply sign in circle]2 = aaT. The maximum likelihood estimators of σk2 based on the kth study data and all data are, respectively,

σ^k2=nk1i=1nk(Ykiβ^kTXkiγ^kTZki)2  and  σ˜k2=nk1i=1nk(Ykiβ˜TXkiγ˜kTZki)2,

where [beta]k and [gamma with circumflex]k are the least-squares estimators of β and γk based on the kth study data, and beta and [gamma with tilde]k are the maximum likelihood estimators of β and γk based on all data. By definition, [beta] = {∑k x2110k ( σ^k2)}−1kx2110k ( σ^k2)[beta]k and vâr([beta]) = {∑k x2110k ( σ^k2)−1. It is easy to show that beta = {∑k x2110k ( σ˜k2)}−1k x2110k ( σ˜k2)[beta]k and vâr(beta) = {∑k x2110k ( σ˜k2)}−1. Thus, [beta] and beta, and their variance estimators, differ only in whether x2110k ( σk2) is evaluated at σ^k2 or σ˜k2. In general, σ^k2 and σ˜k2 are approximately the same, so that the results of meta-analysis using summary statistics and using original data are similar. Under the assumed model, both σ^k2 and σ˜k2 converge to σk2, so that using summary statistics is asymptotically equivalent to using original data. If the assumed model is incorrect, then the two estimators may converge to different constants.

The setting considered by Olkin & Sampson (1998) and Mathew & Nordström (1999) is a special case of our model in which X consists of treatment indicators and Z = 1. In that setting, Dk is a function of group sizes only. Under the assumption that σ12==σK2, one can define [beta] as (∑k Dk)−1k Dk [beta]k, which also turns out to be the expression of beta under the common variance assumption. Thus, using summary statistics is numerically identical to using original data, which is the finding of Olkin & Sampson (1998). Mathew & Nordström (1999) stated that the equivalence continues to hold even if the error variances are unequal. They used the true values of σk2 in their definition of [beta]. Since σk2 need to be estimated from the data, the equivalence holds only asymptotically rather than numerically.

Example 2. Assume that

pr(Yki=1|Xki,Zki)=exp(βTXki+γkTZki)1+exp(βTXki+γkTZki)(k=1,,K;i=1,,ni).

Write θk = (β, γk). The observed profile information matrix for β based on the kth study data is

k(θk)=i=1nkυki(θk)Xki2{i=1nkυki(θk)XkiZkiT}{i=1nkυki(θk)ZkiZkiT}1{i=1nkυki(θk)ZkiXkiT},

where υki(θk)=eβTXki+γkTZki/(1+eβTXki+γkTZki)2. Note that x2110k depends on θk only through the υki (θk). Clearly, υki (θk) = pr(Yki = 1){1 − pr(Yki = 0)}, which is not sensitive to the value of θk unless pr(Yki = 1) is extreme. Thus, the results of meta-analysis using summary statistics and using original data are generally similar, whether the effect sizes are equal or not.

Example 3. The Cox (1972) proportional hazards model specifies that the hazard function of the survival time Yki conditional on covariates Xki takes the form

λ(y|Xki)=λk0(y)exp(βTXki)(k=1,,K;i=1,,ni),

where λk0(·) are arbitrary baseline hazard functions. In the presence of right censoring, the data consist of (ki, Δki, Xki) (k = 1, . . ., K ; i = 1, . . ., nk), where ki = min(Yki, Cki), Δki = I (Yki [less-than-or-eq, slant] Cki), Cki is the censoring time on Yki and I (·) is the indicator function. The observed profile information matrix for β based on the kth study data is k(β)=i=1nkΔkiVk(β;Y˜ki), where

Vk(β;y)=j=1nkI(Y˜kjy)exp(βTXkj)Xkj2j=1nkI(Y˜kjy)exp(βTXkj){j=1nkI(Y˜kjy)exp(βTXkj)Xkjj=1nkI(Y˜kjy)exp(βTXkj)}2.

Here Vk (β ; y) is an empirical covariance matrix of X and is not sensitive to the value of β. Thus, the results of meta-analysis using summary statistics and using original data are similar whether the effect sizes are equal or not.

3. Numerical results

3.1. Simulation studies

We conducted simulation studies to assess how well the asymptotic efficiency results of § 2 approximate realistic situations. We mimicked meta-analysis of randomized clinical trials with a binary outcome and simulated data from the following logistic regression model:

pr(Yki=1|Xki)=exp(αk+βkXki)1+exp(αk+βkXki)(k=1,,K;i=1,,nk),
(7)

where Xki is the treatment indicator, and the proportion of subjects receiving treatment 1 in the kth trial is pk. The first set of simulation studies was focused on K = 2. We set α1 = 0 and α2 = −1 to yield approximately 50% and 30% overall success rates, and chose various values of β1, β2, n1, n2, p1 and p2 to cover a wide range of log odds ratios, sample sizes and treatment assignment ratios. For each combination of the simulation parameters, we generated 1 million datasets; for each dataset, we performed the two types of meta-analysis, i.e. the one based on summary statistics versus the one based on original data. The results are summarized in Table 1.

Table 1.
Meta-analysis based on summary statistics versus original data

When the effect sizes are the same between the two studies, i.e. β1 = β2, both [beta] and beta are nearly unbiased, and their standard errors are close to each other. Thus, the two types of meta-analysis have similar efficiency. To be more specific, [beta] appears to have slightly smaller standard error than beta and is thus a little more efficient in that sense; however, beta is slightly biased upward and its standard error tends to be slightly underestimated, so that the corresponding Wald test tends to be slightly more powerful than that of [beta]. When the effect sizes are unequal between the two studies, i.e. β1β2, the differences between the two methods become more appreciable. In many cases, meta-analysis based on original data is a little more powerful than meta-analysis based on summary statistics. There are also cases in which the latter is slightly more powerful than the former.

In the second set of studies, we simulated K trials of size n from (7) with αk = −1, βk = 1 and pk = 0.5 (k = 1, . . ., K). For both meta-analysis of summary statistics and meta-analysis of original data, the K intercepts might be assumed to be the same or allowed to be different. To impose a common intercept in meta-analysis of summary statistics, we used the bivariate summary statistics for (αk, βk). Table 2 displays the relative efficiency results based on 10 000 replicates with nonzero cell counts. Meta-analysis of summary statistics appears to be as efficient as, or slightly more efficient than, meta-analysis of original data when the two methods make the same modelling assumptions. For n = 10, meta-analysis of original data with a common intercept is a bit more efficient than meta-analysis of summary statistics with different intercepts.

Table 2.
Relative efficiency of using summary statistics versus original data in meta-analysis

3.2. Major depression data

Major depression is a complex common disease with enormous public health significance. The lifetime prevalence of this disorder is approximately 15% and is two-fold higher in women than men. Recently, a genome-wide association study was conducted to identify single nucleotide polymorphisms, SNPs, that are associated with major depression (Sullivan et al., 2009). Using a case-control sample of 1738 cases and 1802 controls, the investigators found strong signals in a region surrounding the gene piccolo, PCLO. The investigators then attempted to replicate the results with five independent case-control samples. For our illustration, we exclude the two replication samples that do not have information on sex, which is an important predictor of major depression. The remaining three replication samples have 1907, 2489 and 2005 subjects, with a total of 3135 cases and 3266 controls.

A total of 30 SNPs in the PCLO region were genotyped in the replication samples. For each SNP, we fit the logistic regression model

pr(Yki=1|Xki,Zki)=exp(αk+βXkiγkZki)1+exp(αk+βXki+γkZki)(k=1,,3;i=1,,nk),
(8)

where Yki is the case-control status of the i th subject in the kth sample, and Xki and Zki are the corresponding genotype score and sex indicator; the genotype score is the number of copies of the less frequent nucleotide of the SNP that the subject carries. The estimates of the genetic effects and their standard error estimates vary substantially among the three replication samples. For meta-analysis of summary statistics, the estimates of the genetic effects pre-adjusted for sex from the three samples are combined according to formula (1).

We allow αk in (8) to be different among the three samples so as to reflect the unequal case-control ratios; γk may be the same or different among the three samples. When γk are allowed to be different in both the meta-analysis based on summary statistics and the meta-analysis based on original data, the two methods yield virtually identical results for the genetic effects of the 30 SNPs: the largest absolute difference between the two sets of log odds ratio estimates is 0.00066, and the largest relative difference between the two sets of standard error estimates is 0.25%. When γk are assumed to be the same among the three replication samples in the meta-analysis of original data but not in the meta-analysis of summary statistics, the differences between the two methods become slightly more noticeable: the largest absolute difference between the two sets of log odds ratio estimates is 0.0045, and the largest relative difference between the two sets of standard error estimates is 0.35%. Incidentally, the covariance estimates between [beta]k and [gamma with circumflex]k are virtually zero, so the differences should be small in light of the results of § 2.2.

4. Remarks

The theoretical results of the present paper are much broader than those of Olkin & Sampson (1998) and Mathew & Nordström (1999), even in the special setting considered by those authors; we have clarified the conditions for the equivalence results stated in those two papers and examined the consequences of violating the underlying assumptions. We have considered more general models for continuous response variables, as well as general parametric and semiparametric models for other response variables.

Our work has important practical implications. There is an ongoing debate on whether the benefits of using original data outweigh the extra cost of taking this approach. The statistical issues surrounding this debate have not been understood well. We have shown theoretically and numerically that there is little or no efficiency gain by analyzing original data. Meta-analysis based on summary statistics reduces resource utilization, simplifies data collection and analysis and avoids the bias and efficiency loss caused by exclusion of studies without original data.

By accessing original data, one can enhance comparability among studies with respect to inclusion/exclusion criteria, definitions of variables, creations of subgroups and adjustments of covariates, ensure estimation of the same parameter by the same statistical method and perform model building and diagnostics. Many of these benefits can still be achieved if all participating investigators follow a common set of guidelines on quality control and statistical analysis and then submit their summary statistics to the meta-analyst. Providing summary statistics is logistically much simpler than transferring original data. Indeed, protection of human subjects and other study policies often prohibit investigators from releasing original data.

One reason for obtaining original data is to model individual-level covariates. It is widely recognized that using study-level summaries of covariates can yield highly biased and inefficient meta-analysis (Berlin et al., 2002; Lambert et al., 2002). We have shown theoretically and numerically that there is no bias or efficiency loss if the effect estimates are properly adjusted for individual-level covariates within each study and then combined via formula (1) or (3). The results of § 2 apply not only to meta-analysis of main effects, but also to meta-analysis of interactions, such as treatment-covariate interactions. If study-specific estimates of interactions are unavailable and one is forced to estimate interactions from study-specific main effects and average covariate values, then there can be serious power loss and bias (Simmonds & Higgins, 2007).

In our context, the asymptotics pertain to individual study sizes nk. For commonly used parametric and semiparametric models, such as linear, logistic and Cox regression models, the asymptotic approximations are accurate even for small nk. When the data for individual studies are very sparse, the parameter estimates may be undefined or unreliable. Then analysis of original data will encounter the same difficulties if it is stratified by studies but will tend to be more stable if it is unstratified. Unstratified analysis is more efficient than stratified analysis, but can be misleading if the underlying populations are different among studies. If individual studies are very small, then model building and diagnostics are possible only by pooling the data.

We have focused on fixed-effects models, which assume a common value for the parameter of main interest among studies. An alternative approach is to employ random-effects models, in which the parameter of main interest is treated as a random variable with different realizations across studies (DerSimonian & Laird, 1986). It is technically more challenging to deal with random-effects models than fixed-effects models. Indeed, the properties of meta-analysis under random-effects models have not been investigated rigorously. Our preliminary investigations reveal that the conclusions of § 2 hold for random-effects models under certain conditions. The results will be communicated in a separate report.

Acknowledgments

This research was supported by the National Institutes of Health, U.S.A. The authors thank the editor and two referees for helpful comments.

Appendix A

Some useful matrix results

Lemma 1. For any matrices A p×q, Bq×q, Cp×q and Dq×q with B > 0 and D > 0,

AB1AT+CD1CT(A+C)(B+D)1(A+C)T.
(A1)

The equality holds if and only if AB−1 = C D−1.

Proof. Since B > 0 and D > 0, we can find a nonsingular matrix P such that B = Pdiag{λ1, . . ., λq}PT and D = Pdiag{μ1, . . ., μq}PT, where λi > 0 and μi > 0 (i = 1, . . ., q). By redefining A as A(PT)−1 and C as C(PT)−1, it suffices to prove the lemma when B = diag{λ1, . . ., λq} and D = diag{μ1, . . ., μq}. Let ai be the i th column of A and ci be the i th column of C. Then (A1) becomes

i=1q(λi1aiaiT+μi1ciciT)i=1q(λi+μi)1(ai+ci)(ai+ci)T.

We wish to show that λi1aiaiT+μi1ciciT(λi+μi)1(ai+ci)(ai+ci)T or equivalently λi2ciciT+μi2aiaiTλiμi(aiciT+ciaiT). The desired inequality holds if, for any x,

(λixTci)2+(μixTai)22λiμi(xTai)(xTci),

which is obvious from the Cauchy–Schwartz inequality. The foregoing inequality becomes an equality if and only if λici = μiai. Thus, the equality in (A1) holds if and only if AB−1 = C D−1.

Appendix B

Consistency of the maximum likelihood estimator of full data under local alternatives

By the profile likelihood theory (Murphy & van der Vaart, 2000), for

logplk(β)=logplk(β^k)12(ββ^k)Tk(βk)(ββ^k)+op(n1/2ββk+1)2

for β in a neighbourhood of the true value of βk. Denote the true value of β by β0. Because [beta]kβk = Op(n−1/2), βkβ0 = O(n−1/2) and x2110k (βk) = x2110k ([beta]k) + op(n), we have

k=1Klogplk(β)=k=1Klogplk(β^k)12k=1K(ββ^k)Tk(β^k)(ββ^k)+op(n1/2ββ0+1)2

for β in a neighbourhood of β0. By definition, [beta] maximizes

12k=1K(ββ^k)Tk(β^k)(ββ^k).

It then follows from the Taylor series expansion that

12k=1K(β^β^k)Tk(β^k)(β^β^k)12k=1K(ββ^k)k(β^k)(ββ^k)+αnβ^β2

for some positive constant α. Thus,

logpl(β^)logpl(β)αnβ^β2+op(nββ02+nβ^β02+1).

Because ‖[beta]β0‖ = O(n−1/2), the foregoing inequality implies that pl([beta]) > pl(β) for any β such that ‖β[beta]‖ = n−1/2M for a large M. Hence, there exists a local maximum within the n−1/2M-neighbourhood of [beta]. We define that estimator as beta and conclude that betaβ0 = Op(n −1/2).

References

  • Berlin JA, Santanna J, Schmid CH, Szczech LA, Feldman HI. Individual patient-versus group-level data meta-regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head. Statist Med. 2002;21:371–87. [PubMed]
  • Chalmers I, Sandercrock P, Wennberg J. The Cochrane collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care. Ann New York Acad Sci. 1993;703:156–65. [PubMed]
  • Cox DR. Regression models and life-tables (with Discussion) J. R. Statist. Soc. B. 1972;34:187–220.
  • DerSimonian R, Laird N. Meta-analysis in clinical trials. Contr. Clin. Trials. 1986;7:177–88. [PubMed]
  • Lambert PC, Sutton AJ, Abrams KR, Jones DR. A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. J Clin Epidemiol. 2002;55:86–94. [PubMed]
  • Lohmueller KE, Pearce CL, Pike, Lander ES, Hirschhorn JN. Meta-analysis of genetic association studies supports a contribution of common variants to susceptibility to common disease. Nature Genet. 2003;33:177–82. [PubMed]
  • Kavvoural FK, Ioannidis JPA. Methods for meta-analysis in genetic association studies: a review of their potential and pitfalls. Hum Genet. 2008;123:1–14. [PubMed]
  • Mathew T, Nordstrom K. On the equivalence of meta-analysis using literature and using individual patient data. Biometrics. 1999;55:1221–3. [PubMed]
  • Murphy SA, van der Vaart AW. On the profile likelihood. J Am Statist Assoc. 2000;95:449–65.
  • Olkin I, Sampson A. Comparison of meta-analysis versus analysis of variance of individual patient data. Biometrics. 1998;54:317–22. [PubMed]
  • Simmonds MC, Higgins JP. Covariate heterogeneity in meta-analysis: criteria for deciding between meta-regression and individual patient data. Statist Med. 2007;26:2982–99. [PubMed]
  • Sullivan PF, de Geus EJC, Willemsen G, James MR, Smit JH, Zandbelt T, Arolt V, Baune BT, Blackwood D, Cichon S, Coventry WL, Dohke M, Farmer A, Fava M, Gordon SD, He Q, Heath A, Heutink P, Holsboer F, Hoogendijk WJ, et al. Genome-wide association for major depressive disorder: a possible role for the presynaptic protein piccolo. Molec Psychiat. 2009;14:359–75. [PMC free article] [PubMed]
  • Sutton AJ, Abrams KR, Jones, Sheldon TA, Song F. Methods for Meta-Analysis in Medical Research. Chichester: Wiley; 2000.
  • The Psychiatric GWAS Consortium Steering Committee A framework for interpreting genome-wide association studies of psychiatric disorders. Molec Psychiat. 2009;14:10–17. [PubMed]
  • Whitehead A. Meta-Analysis of Controlled Clinical Trials. Chichester: Wiley; 2002.
  • Zeggini E, Scott LJ, Saxena R, Voight BF, Marchini JL, Hu T, de Bakker PIW, Abecasis GR, Almgren P, Andersen G, Ardlie K, Boström KB, Bergman RM, Bonnycastle LL, Borch-Johnsen K, Burtt NP, Chen H, Chines PS, Daly MJ, Deodhar P, et al. Meta-analysis of genome-wide association data and large-scale replication identifies additional susceptibility loci for type 2 diabetes. Nature Genet. 2008;40:638–45. [PMC free article] [PubMed]

Articles from Biometrika are provided here courtesy of Oxford University Press
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

  • Meta-Analysis of Sequencing Studies With Heterogeneous Genetic Associations[Genetic epidemiology. 2014]
    Tang ZZ, Lin DY. Genetic epidemiology. 2014 Jul; 38(5)389-401
  • The Power of Meta-Analysis in Genome Wide Association Studies[Annual review of genomics and human genetic...]
    Panagiotou OA, Willer CJ, Hirschhorn JN, Ioannidis JP. Annual review of genomics and human genetics. 2013; 14441-465
  • Meta-Analysis of Genome-Wide Association Studies: No Efficiency Gain in Using Individual Participant Data[Genetic epidemiology. 2010]
    Lin DY, Zeng D. Genetic epidemiology. 2010 Jan; 34(1)10.1002/gepi.20435
  • Integrating and extending cohort studies: lessons from the eXtending Treatments, Education and Networks in Depression (xTEND) study[BMC Medical Research Methodology. ]
    Allen J, Inder KJ, Lewin TJ, Attia JR, Kay-Lambkin FJ, Baker AL, Hazell T, Kelly BJ. BMC Medical Research Methodology. 13122
  • Meta-analysis of Gene-Level Associations for Rare Variants Based on Single-Variant Statistics[American Journal of Human Genetics. 2013]
    Hu YJ, Berndt SI, Gustafsson S, Ganna A, Genetic Investigation of ANthropometric Traits (GIANT) ConsortiumBerndtSonja I.SIGustafssonStefanSMägiReedikRGannaAndreaAWheelerEleanorEFeitosaMary F.MFJusticeAnne E.AEMondaKeri L.KLCroteau-ChonkaDamien C.DCDayFelix R.FREskoTõnuTFallToveTFerreiraTeresaTGentiliniDavideDJacksonAnne U.AULuanJian’anJRandallJoshua C.JCVedantamSailajaSWillerCristen J.CJWinklerThomas W.TWWoodAndrew R.ARWorkalemahuTsegaselassieTHuYi-JuanYJLeeSang HongSHLiangLimingLLinDan-YuDYMinJosine L.JLNealeBenjamin M.BMThorleifssonGudmarGYangJianJAlbrechtEvaEAminNajafNBragg-GreshamJennifer L.JLCadbyGemmaGden HeijerMartinMEklundNiinaNFischerKristaKGoelAnujAHottengaJouke-JanJJHuffmanJennifer E.JEJarickIvonneIJohanssonÅsaÅJohnsonTobyTKanoniStavroulaSKleberMarcus E.MEKönigInke R.IRKristianssonKatiKKutalikZoltánZLaminaClaudiaCLecoeurCecileCLiGuoGManginoMassimoMMcArdleWendy L.WLMedina-GomezCarolinaCMüller-NurasyidMartinaMNgwaJulius S.JSNolteIlja M.IMPaternosterLaviniaLPechlivanisSonaliSPerolaMarkusMPetersMarjolein J.MJPreussMichaelMRoseLynda M.LMShiJianxinJShunginDmitryDSmithAlbert VernonAVStrawbridgeRona J.RJSurakkaIdaITeumerAlexanderATripMieke D.MDTyrerJonathanJVan Vliet-OstaptchoukJana V.JVVandenputLiesbethLWaiteLindsay L.LLZhaoJing HuaJHAbsherDevinDAsselbergsFolkert W.FWAtalayMustafaMAttwoodAntony P.APBalmforthAnthony J.AJBasartHannekeHBeilbyJohnJBonnycastleLori L.LLBrambillaPaoloPBruinenbergMarcelMCampbellHarryHChasmanDaniel I.DIChinesPeter S.PSCollinsFrancis S.FSConnellJohn M.JMCooksonWilliamWde FaireUlfUde VegtFemmieFDeiMarianoMDimitriouMariaMEdkinsSarahSEstradaKarolKEvansDavid M.DMFarrallMartinMFerrarioMarco M.MMFerrièresJeanJFrankeLudeLFrauFrancescaFGejmanPablo V.PVGrallertHaraldHGrönbergHenrikHGudnasonVilmundurVHallAlistair S.ASHallPerPHartikainenAnna-LiisaALHaywardCarolineCHeard-CostaNancy L.NLHeathAndrew C.ACHebebrandJohannesJHomuthGeorgGHuFrank B.FBHuntSarah E.SEHyppönenElinaEIribarrenCarlosCJacobsKevin B.KBJanssonJohn-OlovJOJulaAnttiAKähönenMikaMKathiresanSekarSKeeFrankFKhawKay-TeeKTKivimakiMikaMKoenigWolfgangWKrajaAldi T.ATKumariMeenaMKuulasmaaKariKKuusistoJohannaJLaitinenJaana H.JHLakkaTimo A.TALangenbergClaudiaCLaunerLenore J.LJLindLarsLLindströmJaanaJLiuJianjunJLiuzziAntonioALokkiMarja-LiisaMLLorentzonMattiasMMaddenPamela A.PAMagnussonPatrik K.PKManuntaPaoloPMarekDianaDMärzWinfriedWLeachIrene MateoIMMcKnightBarbaraBMedlandSarah E.SEMihailovEvelinEMilaniLiliLMontgomeryGrant W.GWMooserVincentVMühleisenThomas W.TWMunroePatricia B.PBMuskArthur W.AWNarisuNarisuNNavisGerjanGNicholsonGeorgeGNohrEllen A.EAOngKen K.KKOostraBen A.BAPalmerColin N.A.CNPalotieAarnoAPedenJohn F.JFPedersenNancyNPetersAnnetteAPolasekOzrenOPoutaAnneliAPramstallerPeter P.PPProkopenkoIngaIPütterCarolinCRadhakrishnanAparnaARaitakariOlliORendonAugustoARivadeneiraFernandoFRudanIgorISaaristoTimo E.TESambrookJennifer G.JGSandersAlan R.ARSannaSerenaSSaramiesJoukoJSchipfSabineSSchreiberStefanSSchunkertHeribertHShinSo-YounSYSignoriniStefanoSSinisaloJuhaJSkrobekBorisBSoranzoNicoleNStančákováAlenaAStarkKlausKStephensJonathan C.JCStirrupsKathleenKStolkRonald P.RPStumvollMichaelMSwiftAmy J.AJTheodorakiEirini V.EVThorandBarbaraBTregouetDavid-AlexandreDATremoliElenaEVan der KlauwMelanie M.MMvan MeursJoyce B.J.JBVermeulenSita H.SHViikariJormaJVirtamoJarmoJVitartVeroniqueVWaeberGérardGWangZhaomingZWidénElisabethEWildSarah H.SHWillemsenGonnekeGWinkelmannBernhard R.BRWittemanJacqueline C.M.JCWolffenbuttelBruce H.R.BHWongAndrewAWrightAlan F.AFZillikensM. CarolaMCAmouyelPhilippePBoehmBernhard O.BOBoerwinkleEricEBoomsmaDorret I.DICaulfieldMark J.MJChanockStephen J.SJCupplesL. AdrienneLACusiDanieleDDedoussisGeorge V.GVErdmannJeanetteJErikssonJohan G.JGFranksPaul W.PWFroguelPhilippePGiegerChristianCGyllenstenUlfUHamstenAndersAHarrisTamara B.TBHengstenbergChristianCHicksAndrew A.AAHingoraniAroonAHinneyAnkeAHofmanAlbertAHovinghKees G.KGHveemKristianKIlligThomasTJarvelinMarjo-RiittaMRJöckelKarl-HeinzKHKeinanen-KiukaanniemiSirkka M.SMKiemeneyLambertus A.LAKuhDianaDLaaksoMarkkuMLehtimäkiTerhoTLevinsonDouglas F.DFMartinNicholas G.NGMetspaluAndresAMorrisAndrew D.ADNieminenMarkku S.MSNjølstadIngerIOhlssonClaesCOldehinkelAlbertine J.AJOuwehandWillem H.WHPalmerLyle J.LJPenninxBrendaBPowerChrisCProvinceMichael A.MAPsatyBruce M.BMQiLuLRauramaaRainerRRidkerPaul M.PMRipattiSamuliSSalomaaVeikkoVSamaniNilesh J.NJSniederHaroldHSørensenThorkild I.A.TISpectorTimothy D.TDStefanssonKariKTönjesAnkeATuomilehtoJaakkoJUitterlindenAndré G.AGUusitupaMattiMvan der HarstPimPVollenweiderPeterPWallaschofskiHenriHWarehamNicholas J.NJWatkinsHughHWichmannH.-ErichHEWilsonJames F.JFAbecasisGoncalo R.GRAssimesThemistocles L.TLBarrosoInêsIBoehnkeMichaelMBoreckiIngrid B.IBDeloukasPanosPFoxCaroline S.CSFraylingTimothyTGroopLeif C.LCHaritunianTalinTHeidIris M.IMHunterDavidDKaplanRobert C.RCKarpeFredrikFMoffattMiriamMMohlkeKaren L.KLO’ConnellJeffrey R.JRPawitanYudiYSchadtEric E.EESchlessingerDavidDSteinthorsdottirValgerdurVStrachanDavid P.DPThorsteinsdottirUnnurUvan DuijnCornelia M.CMVisscherPeter M.PMDi BlasioAnna MariaAMHirschhornJoel N.JNLindgrenCecilia M.CMMorrisAndrew P.APMeyreDavidDScheragAndréAMcCarthyMark I.MISpeliotesElizabeth K.EKNorthKari E.KELoosRuth J.F.RJIngelssonErikE, Hirschhorn J, North KE, Ingelsson E, Lin DY. American Journal of Human Genetics. 2013 Aug 8; 93(2)236-248
See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...