Probability of replication (p_rep) as a function of the number of observations and measured effect size, d₁′. The functions in each panel show p_rep for values of d₁′ increasing in steps of 0.1, from 0.10 (lowest curve) to 1.0 (highest curve). The dashed lines show the combination of effect size and n necessary to reject a null hypothesis of no difference between the means of the experimental and control groups (i.e., μ_E − μ_C = 0) using a two-tailed t test with α = .05. When realization variance, σ_δ², is 0 (left panel), replicability functions asymptote at 1.0. For a one-tailed test, the dashed line drops to .88. When realization variance is 0.08 (right panel), the median for social psychological research, replicability functions asymptote below 1.0. As n approaches infinity, the t-test criterion falls to an asymptote of .5.

Sampling distributions of effect size (d). The top panel shows a distribution for a population effect size of δ = 0.1; the experiment yielded an effect size of 0.3, and thus had a sampling error Δ = d₁ ′ − δ = 0.2. The middle panel shows the probability of a replication as the area under the sampling distribution to the right of 0, given knowledge that δ = 0.1. The bottom panel shows the posterior predictive density of effect size in replication. Absent knowledge of δ, the probability of replication is predicted as the area to the right of 0.

Complementarity of p_rep and p. The top panel shows sampling distributions for d₁′ given the null (left) and for d₂′ given d₁ (right). The small black area gives the probability of finding a statistic more extreme than d₁ if the null were true. The large shaded area gives the probability of finding supportive evidence in an equipotent replication. In the bottom panel, p_rep is plotted against the p values calculated for the normal distribution under the null hypothesis with d = 0.1, 0.2, . . . , 1.0, and n ranging from 10 to 80; p_rep is calculated from Equations 3, 5, and 6. The function is described in the appendix.