Pruning Maxout Network

• 1、Sino-France Engineering School,Beihang university,Beijing,100191
• 2、Department of Mathematics and Computer Science,école Centrale de Lyon,Lyon,69130,France

Abstract：Recently the maxout network is widely applied in computer vison and speech recognition and proved to be a state of the art method. Like Dropout, a maxout unit takes the max output of its candidates, so the maxout network aggregates actually a set of its sub-network. However, the large number of units in maxout layers leads to the increase of training time, and certains units aren't well-trained. Besides, the increase of the parameters could cause the over-fitting problems. In this paper we analyze the redundancy of the Maxout network and propose a method to prune the redundant units. The experiments prove that the pruned model can keep the performance of the original model, while reducing the number of parameters about 50% to 60% . In addition, the pruned model performs better than the model which has the same numbers of parameters as the pruned model, which means that pruning a complex model can not only reduce the storage space, but also give a good initialization of parameters for smaller models.

Keywords： convolutional neural network maxout network pruning method