一种基于注意力机制的卷积神经网络剪枝方法
首发时间:2019-03-29
摘要:卷积神经网络剪枝技术可以有效减少神经网络的内存占用和运行耗时,有利于神经网络在资源有限的设备上部署。剪枝研究中,如何衡量参数重要性是核心问题,参数重要性直接决定哪些参数被优先移除。本文提出一种基于注意力机制的卷积神经网络剪枝方法,该方法借助注意力模块,自适应地学习卷积网络中同层滤波器输出通道的权重,并以此计算滤波器重要性,在滤波器级别指导网络剪枝。实验结果表明,本方法在相同剪枝比例下可以获得更高的准确率,并且不依赖特定的底层运算库或硬件设备。
For information in English, please click here
A Convolutional Neural Network Pruning Method Based on Attention Mechanism
Abstract:Convolutional neural network pruning technology can effectively reduce the memory and running time cost of the neural network, which is conducive to the deployment of the neural network on equipment with limited resources. In pruning research, how to measure the importance of parameters is the core issue. The importance of parameters directly determines which parameters can be removed first. In this paper, a convolutional neural network pruning method based on attention mechanism is proposed. With the help of attention module, the method adaptively learns the weight of the output channel of the same-layer filter in convolutional network, and calculates the importance of the filter to guide network pruning at the filter level. The experimental results show that this method can achieve higher accuracy under the same pruning ratio, and does not depend on specific underlying operation libraries or hardware devices.
Keywords: Convolutional Neural Network Pruning Attention Mechanism
基金:
引用
No.****
同行评议
勘误表
一种基于注意力机制的卷积神经网络剪枝方法
评论
全部评论0/1000