Format

Send to

Choose Destination
J Clin Epidemiol. 2015 Sep;68(9):1046-58. doi: 10.1016/j.jclinepi.2015.05.029. Epub 2015 Jun 6.

Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations.

Author information

1
Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck St., Room 314A, Boston, MA 02115, USA.
2
Melbourne School of Population and Global Health, Level 4, 207 Bouverie St., The University of Melbourne, Victoria 3010, Australia.
3
Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck St., Room 314A, Boston, MA 02115, USA; Department of Medicine, Stanford Prevention Research Center, Stanford University School of Medicine, Medical School Office Building, Room X306, 1265 Welch Rd, Stanford, CA 94305, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA 94305, USA; Department of Health Research and Policy, Stanford University School of Medicine, Stanford, CA 94305, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA 94305, USA. Electronic address: jioannid@stanford.edu.

Abstract

OBJECTIVES:

Model specification-what adjusting variables are analytically modeled-may influence results of observational associations. We present a standardized approach to quantify the variability of results obtained with choices of adjustments called the "vibration of effects" (VoE).

STUDY DESIGN AND SETTING:

We estimated the VoE for 417 clinical, environmental, and physiological variables in association with all-cause mortality using National Health and Nutrition Examination Survey data. We selected 13 variables as adjustment covariates and computed 8,192 Cox models for each of 417 variables' associations with all-cause mortality.

RESULTS:

We present the VoE by assessing the variance of the effect size and in the -log10(P-value) obtained by different combinations of adjustments. We present whether there are multimodality patterns in effect sizes and P-values and the trajectory of results with increasing adjustments. For 31% of the 417 variables, we observed a Janus effect, with the effect being in opposite direction in the 99th versus the 1st percentile of analyses. For example, the vitamin E variant α-tocopherol had a VoE that indicated higher and lower risk for mortality.

CONCLUSION:

Estimating VoE offers empirical estimates of associations are under different model specifications. When VoE is large, claims for observational associations should be very cautious.

KEYWORDS:

Biostatistics; Confounding; Environment-wide association study; Model specification; Observational association; Vibration of effects

PMID:
26279400
PMCID:
PMC4555355
DOI:
10.1016/j.jclinepi.2015.05.029
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Elsevier Science Icon for PubMed Central
Loading ...
Support Center