Send to

Choose Destination

See 1 citation:

Mach Learn. 2015 Apr 1;99(1):137-163.

A unified statistical approach to non-negative matrix factorization and probabilistic latent semantic indexing.

Author information

Department of Biostatistics and Bioinformatics, Fox Chase Cancer Center, Temple University Health System, Philadelphia, PA 19111, USA.
3M Health Information Systems, Bethesda, MD 20814, USA.
Division of Statistics, Northern Illinois University, DeKalb, IL 60115, USA.


Non-negative matrix factorization (NMF) is a powerful machine learning method for decomposing a high-dimensional nonnegative matrix V into the product of two nonnegative matrices, W and H, such that VW H. It has been shown to have a parts-based, sparse representation of the data. NMF has been successfully applied in a variety of areas such as natural language processing, neuroscience, information retrieval, image processing, speech recognition and computational biology for the analysis and interpretation of large-scale data. There has also been simultaneous development of a related statistical latent class modeling approach, namely, probabilistic latent semantic indexing (PLSI), for analyzing and interpreting co-occurrence count data arising in natural language processing. In this paper, we present a generalized statistical approach to NMF and PLSI based on Renyi's divergence between two non-negative matrices, stemming from the Poisson likelihood. Our approach unifies various competing models and provides a unique theoretical framework for these methods. We propose a unified algorithm for NMF and provide a rigorous proof of monotonicity of multiplicative updates for W and H. In addition, we generalize the relationship between NMF and PLSI within this framework. We demonstrate the applicability and utility of our approach as well as its superior performance relative to existing methods using real-life and simulated document clustering data.


Biomedical informatics; EM algorithm; Nonnegative matrix factorization; Probabilistic latent semantic indexing; Renyi's divergence; λ-log-likelihood

Supplemental Content

Full text links

Icon for PubMed Central
Loading ...
Support Center