Send to

Choose Destination
J Clin Epidemiol. 2015 Sep;68(9):1046-58. doi: 10.1016/j.jclinepi.2015.05.029. Epub 2015 Jun 6.

Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations.

Author information

Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck St., Room 314A, Boston, MA 02115, USA.
Melbourne School of Population and Global Health, Level 4, 207 Bouverie St., The University of Melbourne, Victoria 3010, Australia.
Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck St., Room 314A, Boston, MA 02115, USA; Department of Medicine, Stanford Prevention Research Center, Stanford University School of Medicine, Medical School Office Building, Room X306, 1265 Welch Rd, Stanford, CA 94305, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA 94305, USA; Department of Health Research and Policy, Stanford University School of Medicine, Stanford, CA 94305, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA 94305, USA. Electronic address:



Model specification-what adjusting variables are analytically modeled-may influence results of observational associations. We present a standardized approach to quantify the variability of results obtained with choices of adjustments called the "vibration of effects" (VoE).


We estimated the VoE for 417 clinical, environmental, and physiological variables in association with all-cause mortality using National Health and Nutrition Examination Survey data. We selected 13 variables as adjustment covariates and computed 8,192 Cox models for each of 417 variables' associations with all-cause mortality.


We present the VoE by assessing the variance of the effect size and in the -log10(P-value) obtained by different combinations of adjustments. We present whether there are multimodality patterns in effect sizes and P-values and the trajectory of results with increasing adjustments. For 31% of the 417 variables, we observed a Janus effect, with the effect being in opposite direction in the 99th versus the 1st percentile of analyses. For example, the vitamin E variant α-tocopherol had a VoE that indicated higher and lower risk for mortality.


Estimating VoE offers empirical estimates of associations are under different model specifications. When VoE is large, claims for observational associations should be very cautious.


Biostatistics; Confounding; Environment-wide association study; Model specification; Observational association; Vibration of effects

[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Elsevier Science Icon for PubMed Central
Loading ...
Support Center