Logo of ejbiosysbioJournal's HomeManuscript SubmissionSpringerOpen.comRegisterThis article
EURASIP J Bioinform Syst Biol. 2007; 2007(1): 79879.
Published online 2007 Jun 24. doi:  10.1155/2007/79879
PMCID: PMC3171353

Information-Theoretic Inference of Large Transcriptional Regulatory Networks


The paper presents MRNET, an original method for inferring genetic networks from microarray data. The method is based on maximum relevance/minimum redundancy (MRMR), an effective information-theoretic technique for feature selection in supervised learning. The MRMR principle consists in selecting among the least redundant variables the ones that have the highest mutual information with the target. MRNET extends this feature selection principle to networks in order to infer gene-dependence relationships from microarray data. The paper assesses MRNET by benchmarking it against RELNET, CLR, and ARACNE, three state-of-the-art information-theoretic methods for large (up to several thousands of genes) network inference. Experimental results on thirty synthetically generated microarray datasets show that MRNET is competitive with these methods.


  • van Someren EP, Wessels LFA, Backer E, Reinders MJT. Genetic network modeling. Pharmacogenomics. 2002;3(4):507–525. doi: 10.1517/14622416.3.4.507. [PubMed] [Cross Ref]
  • Gardner TS, Faith JJ. Reverse-engineering transcription control networks. Physics of Life Reviews. 2005;2(1):65–88. doi: 10.1016/j.plrev.2005.01.001. [PubMed] [Cross Ref]
  • Chow C, Liu C. Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory. 1968;14(3):462–467. doi: 10.1109/TIT.1968.1054142. [Cross Ref]
  • Butte AJ, Kohane IS. Mutual information relevance networks: functional genomic clustering using pairwise entropy measurements. Pacific Symposium on Biocomputing. 2000. pp. 418–429. [PubMed]
  • Margolin AA, Nemenman I, Basso K. et al. ARACNE: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context. BMC Bioinformatics. 2006;7(1):S7. doi: 10.1186/1471-2105-7-S1-S7. [PMC free article] [PubMed] [Cross Ref]
  • Faith JJ, Hayete B, Thaden JT. et al. Large-scale mapping and validation of Escherichia coli transcriptional regulation from a compendium of expression profiles. PLoS Biology. 2007;5(1):e8. doi: 10.1371/journal.pbio.0050008. [PMC free article] [PubMed] [Cross Ref]
  • Pearl J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible. Morgan Kaufmann, San Fransisco, Calif, USA; 1988.
  • Cheng J, Greiner R, Kelly J, Bell D, Liu W. Learning Bayesian networks from data: an information-theory based approach. Artificial Intelligence. 2002;137(1-2):43–90. doi: 10.1016/S0004-3702(02)00191-1. [Cross Ref]
  • Schneidman E, Still S, Berry MJ II, Bialek W. Network information and connected correlations. Physical Review Letters. 2003;91(23):4 pages. [PubMed]
  • Nemenman I. Tech. Rep. NSF-KITP-04-54. KITP, UCSB, Santa Barbara, Calif, USA; 2004. Multivariate dependence, and genetic network inference.
  • Tourassi GD, Frederick ED, Markey MK, Floyd CE Jr.. Application of the mutual information criterion for feature selection in computer-aided diagnosis. Medical Physics. 2001;28(12):2394–2402. doi: 10.1118/1.1418724. [PubMed] [Cross Ref]
  • Ding C, Peng H. Minimum redundancy feature selection from microarray gene expression data. Journal of Bioinformatics and Computational Biology. 2005;3(2):185–205. doi: 10.1142/S0219720005001004. [PubMed] [Cross Ref]
  • Meyer PE, Bontempi G. In: Applications of Evolutionary Computing: EvoWorkshops, Lecture Notes in Computer Science. Rothlauf F, Branke J, Cagnoniet S, et al., editor. Vol. 3907. Springer, Berlin, Germany; 2006. On the use of variable complementarity for feature selection in cancer classification; pp. 91–102. [Cross Ref]
  • Meyer PE, Kontos K, Bontempi G. Biological network inference using redundancy analysis. Proceedings of the 1st International Conference on Bioinformatics Research and Development (BIRD '07), Berlin, Germany, March 2007. pp. 916–927.
  • Butte AJ, Tamayo P, Slonim D, Golub TR, Kohane IS. Discovering functional relationships between RNA expression and chemotherapeutic susceptibility using relevance networks. Proceedings of the National Academy of Sciences of the United States of America. 2000;97(22):12182–12186. doi: 10.1073/pnas.220392197. [PMC free article] [PubMed] [Cross Ref]
  • Cover TM, Thomas JA. Elements of Information Theory. John Wiley & Sons, New York, NY, USA; 1990.
  • Merz P, Freisleben B. Greedy and local search heuristics for unconstrained binary quadratic programming. Journal of Heuristics. 2002;8(2):197–213. doi: 10.1023/A:1017912624016. [Cross Ref]
  • Rogers S, Girolami M. A Bayesian regression approach to the inference of regulatory networks from gene expression data. Bioinformatics. 2005;21(14):3131–3137. doi: 10.1093/bioinformatics/bti487. [PubMed] [Cross Ref]
  • van den Bulcke T, van Leemput K, Naudts B. et al. SynTReN: a generator of synthetic gene expression data for design and analysis of structure learning algorithms. BMC Bioinformatics. 2006;7:43. doi: 10.1186/1471-2105-7-43. [PMC free article] [PubMed] [Cross Ref]
  • Paninski L. Estimation of entropy and mutual information. Neural Computation. 2003;15(6):1191–1253. doi: 10.1162/089976603321780272. [Cross Ref]
  • Beirlant J, Dudewica EJ, Gyofi L, van der Meulen E. Nonparametric entropy estimation: an overview. Journal of Statistics. 1997;6(1):17–39.
  • Dougherty J, Kohavi R, Sahami M. Supervised and unsupervised discretization of continuous features. Proceedings of the 12th International Conference on Machine Learning (ML '95), Lake Tahoe, Calif, USA, July 1995. pp. 194–202.
  • Provost FJ, Fawcett T, Kohavi R. Proceedings of the 15th International Conference on Machine Learning (ICML '98), Madison, Wis, USA, July 1998. Morgan Kaufmann; The case against accuracy estimation for comparing induction algorithms; pp. 445–453.
  • Bockhorst J, Craven M. In: Advances in Neural Information Processing Systems 17. Saul LK, Weiss Y, Bottou L, editor. MIT Press, Cambridge, Mass, USA; 2005. Markov networks for detecting overlapping elements in sequence data; pp. 193–200.
  • Dietterich TG. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation. 1998;10(7):1895–1923. doi: 10.1162/089976698300017197. [PubMed] [Cross Ref]
  • Hwang KB, Lee JW, Chung S-W, Zhang B-T. Construction of large-scale Bayesian networks by local to global search. Proceedings of the 7th Pacific Rim International Conference on Artificial Intelligence (PRICAI '02), Tokyo, Japan, August 2002. pp. 375–384.
  • Tsamardinos I, Aliferis C, Statnikov A. Algorithms for large scale markov blanket discovery. Proceedings of the 16th International Florida Artificial Intelligence Research Society Conference (FLAIRS '03), St. Augustine, Fla, USA, May 2003. pp. 376–381.
  • Tsamardinos I, Aliferis C. Towards principled feature selection: relevancy, filters and wrappers. Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics (AI&Stats '03), Key West, Fla, USA. January 2003.

Articles from EURASIP Journal on Bioinformatics and Systems Biology are provided here courtesy of BioMed Central
PubReader format: click here to try


Save items

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...


  • PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...