<PubmedArticle>
    <MedlineCitation Status="PubMed-not-MEDLINE" Owner="NLM">
        <PMID Version="1">22509963</PMID>
        <DateCompleted>
            <Year>2012</Year>
            <Month>10</Month>
            <Day>17</Day>
        </DateCompleted>
        <DateRevised>
            <Year>2012</Year>
            <Month>06</Month>
            <Day>13</Day>
        </DateRevised>
        <Article PubModel="Print-Electronic">
            <Journal>
                <ISSN IssnType="Electronic">1530-888X</ISSN>
                <JournalIssue CitedMedium="Internet">
                    <Volume>24</Volume>
                    <Issue>8</Issue>
                    <PubDate>
                        <Year>2012</Year>
                        <Month>Aug</Month>
                    </PubDate>
                </JournalIssue>
                <Title>Neural computation</Title>
                <ISOAbbreviation>Neural Comput</ISOAbbreviation>
            </Journal>
            <ArticleTitle>An efficient learning procedure for deep Boltzmann machines.</ArticleTitle>
            <Pagination>
                <MedlinePgn>1967-2006</MedlinePgn>
            </Pagination>
            <ELocationID EIdType="doi" ValidYN="Y">10.1162/NECO_a_00311</ELocationID>
            <Abstract>
                <AbstractText>We present a new learning algorithm for Boltzmann machines that contain many layers of hidden variables. Data-dependent statistics are estimated using a variational approximation that tends to focus on a single mode, and data-independent statistics are estimated using persistent Markov chains. The use of two quite different techniques for estimating the two types of statistic that enter into the gradient of the log likelihood makes it practical to learn Boltzmann machines with multiple hidden layers and millions of parameters. The learning can be made more efficient by using a layer-by-layer pretraining phase that initializes the weights sensibly. The pretraining also allows the variational inference to be initialized sensibly with a single bottom-up pass. We present results on the MNIST and NORB data sets showing that deep Boltzmann machines learn very good generative models of handwritten digits and 3D objects. We also show that the features discovered by deep Boltzmann machines are a very effective way to initialize the hidden layers of feedforward neural nets, which are then discriminatively fine-tuned.</AbstractText>
            </Abstract>
            <AuthorList CompleteYN="Y">
                <Author ValidYN="Y">
                    <LastName>Salakhutdinov</LastName>
                    <ForeName>Ruslan</ForeName>
                    <Initials>R</Initials>
                    <AffiliationInfo>
                        <Affiliation>Department of Statistics, University of Toronto, Toronto, Ontario M5S 3G3, Canada. rsalakhu@utstat.toronto.edu</Affiliation>
                    </AffiliationInfo>
                </Author>
                <Author ValidYN="Y">
                    <LastName>Hinton</LastName>
                    <ForeName>Geoffrey</ForeName>
                    <Initials>G</Initials>
                </Author>
            </AuthorList>
            <Language>eng</Language>
            <PublicationTypeList>
                <PublicationType UI="D016428">Journal Article</PublicationType>
                <PublicationType UI="D013485">Research Support, Non-U.S. Gov't</PublicationType>
            </PublicationTypeList>
            <ArticleDate DateType="Electronic">
                <Year>2012</Year>
                <Month>04</Month>
                <Day>17</Day>
            </ArticleDate>
        </Article>
        <MedlineJournalInfo>
            <Country>United States</Country>
            <MedlineTA>Neural Comput</MedlineTA>
            <NlmUniqueID>9426182</NlmUniqueID>
            <ISSNLinking>0899-7667</ISSNLinking>
        </MedlineJournalInfo>
    </MedlineCitation>
    <PubmedData>
        <History>
            <PubMedPubDate PubStatus="entrez">
                <Year>2012</Year>
                <Month>4</Month>
                <Day>19</Day>
                <Hour>6</Hour>
                <Minute>0</Minute>
            </PubMedPubDate>
            <PubMedPubDate PubStatus="pubmed">
                <Year>2012</Year>
                <Month>4</Month>
                <Day>19</Day>
                <Hour>6</Hour>
                <Minute>0</Minute>
            </PubMedPubDate>
            <PubMedPubDate PubStatus="medline">
                <Year>2012</Year>
                <Month>4</Month>
                <Day>19</Day>
                <Hour>6</Hour>
                <Minute>1</Minute>
            </PubMedPubDate>
        </History>
        <PublicationStatus>ppublish</PublicationStatus>
        <ArticleIdList>
            <ArticleId IdType="pubmed">22509963</ArticleId>
            <ArticleId IdType="doi">10.1162/NECO_a_00311</ArticleId>
        </ArticleIdList>
    </PubmedData>
</PubmedArticle>