Format

Send to

Choose Destination
See comment in PubMed Commons below
IEEE Trans Pattern Anal Mach Intell. 2013 Feb;35(2):398-410.

Learning multivariate distributions by competitive assembly of marginals.

Author information

1
Department of Applied Mathematics and Statistics, Center for Imaging Science and Institute for Computational Medicine, Johns Hopkins University, Clark Hall, 3400 N. Charles St., Baltimore, MD 21218, USA. sanchez@cis.jhu.edu

Abstract

We present a new framework for learning high-dimensional multivariate probability distributions from estimated marginals. The approach is motivated by compositional models and Bayesian networks, and designed to adapt to small sample sizes. We start with a large, overlapping set of elementary statistical building blocks, or "primitives," which are low-dimensional marginal distributions learned from data. Each variable may appear in many primitives. Subsets of primitives are combined in a Lego-like fashion to construct a probabilistic graphical model; only a small fraction of the primitives will participate in any valid construction. Since primitives can be precomputed, parameter estimation and structure search are separated. Model complexity is controlled by strong biases; we adapt the primitives to the amount of training data and impose rules which restrict the merging of them into allowable compositions. The likelihood of the data decomposes into a sum of local gains, one for each primitive in the final structure. We focus on a specific subclass of networks which are binary forests. Structure optimization corresponds to an integer linear program and the maximizing composition can be computed for reasonably large numbers of variables. Performance is evaluated using both synthetic data and real datasets from natural language processing and computational biology.

PMID:
22529323
DOI:
10.1109/TPAMI.2012.96
[Indexed for MEDLINE]
PubMed Commons home

PubMed Commons

0 comments

    Supplemental Content

    Full text links

    Icon for IEEE Engineering in Medicine and Biology Society
    Loading ...
    Support Center