Introduction of Restricted Boltzmann Machines

• 1、School of Mathematics and Statistics, Xi\'an Jiaotong University, Xi\'an 710049
• 2、 School of Mathematics and Statistics, Xi\'an Jiaotong University, Xi\'an 710049
• 3、 School of Mechanical Engineering, Xi\'an Jiaotong University, Xi\'an 710049

Abstract：A Restricted Boltzmann Machine (RBM) is a particular type of random neural network model which has two-layer architecture, symmetric connections and no self-feedback. The two layers in an RBM are fully connected but there are no connections within the same layer. Recently, with the advent of a fast learning algorithm for RBM (i.e., contrastive divergence), the machine learning community set off a surge to study the theory and applications of RBM since it has many advantages. For example, RBM is an effective method to detect features. When a feed-forward neural network is initialized with an RBM, its generalization capability can be significantly improved. A deep belief network composed of several RBMs can detect more abstract features. Due to the advantages and wide applications of RBM, this paper attempts to provide a started guide for novice. It presents a detailed introduction of basic RBM model, its representative learning algorithm, parametric settings, evaluation methods, its variants and etc. Finally, some research directions of RBM that are deserved to be further studied are discussed.

Keywords： Machine learning Deep learning Restricted Boltzmann machine Contrastive divergence Gibbs sampling.