A Quantization Training Algorithm of Adaptive Learning Quantization Scale Fators

doi:10.16182/j.issn1004731x.joss.21-0175

Abstract

Abstract:

Deep neural network model is difficult to effectively deploy in embedded terminals due to its excessive number of components, andone of the solutions is model miniaturization (such as model quantization, knowledge distillation, etc.). To address this problem, a quantization training algorithm (referred to as LSQ-BN algorithm) based on adaptive learning of quantizationscale factors with BN folding is proposed.A single CNN (convolutional neural) is usedtoconstruct BN folding and achieve BN and CNN fusion. During the process of quantitative training,the quantization scale factors are set as model parameters. An adaptive quantizationscale factor initialization scheme is proposed to solve the problem of difficult initialization of quantizationscale factors.The experimental results show that the precision of the quantized model is almost the same as that of the FP32 prefabricated model when the weight and activation are both 8bit quantization. When the weight is 4 bit quantization and the activation is 8bit quantization, the precision loss of the quantization model is within 3%. Therefore, LSQ-BN proposed in this paper is an excellent model quantization algorithm.

Key words: BN folding, CNN convolution, adaptive initialization, model quantization scale-factor

CLC Number:

TP391

Hui Nie, Kangshun Li, Yang Su. A Quantization Training Algorithm of Adaptive Learning Quantization Scale Fators[J]. Journal of System Simulation, 2022, 34(7): 1639-1650.

Figures/Tables 17

Table 1

Fig. 1

Table 2

Quantization range

类别	符号项	取值(b表示给定的量化数据位长)
对于无符号的bit位量化	$Q P$	$Q P = 2 b - 1$
对于无符号的bit位量化	$Q N$	$Q N = 0$
对于有符号的bit位量化	$Q P$	$Q P = 2 b - 1 - 1$
对于有符号的bit位量化	$Q N$	$Q N = 2 b - 1 - 1$

Table 2

Fig. 2

Table 3

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 4

Table 5

Table 6

Fig. 9

Fig. 10

Fig. 11

References 21

1	Zhang L, Li K, Qi Y, et al. Local Feature Extracted by the Improved Bag of Features Method for Person re-Identification[J]. Neurocomputing (S0925-2312), 2021, 458: 690-700.
2	Tan Zhiping, Li Kangshun, Wang Yi. Differential Evolution with Adaptive Mutation Strategy Based on Fitness Landscape Analysis[J]. Information Sciences(S0020-0255), 2021,549:142-163.
3	阴敬方, 朱登明, 石敏, 等. 基于引导对抗网络的人体深度图像修补方法[J]. 系统仿真学报, 2020,32(7): 1312-1321.
	Yin Jingfang, Zhu Dengming, Shi Min, et al. Human Depth Image Repairing Method Based on Guided adversation Network[J]. Journal of System Simulation, 2020, 32(7): 1312-1321.
4	黄欣, 方钰, 顾梦丹. 基于卷积神经网络的X线胸片疾病分类研究[J]. 系统仿真学报, 2020, 32(6): 1188-1194.
	Huang Xin, Fang Yu, Gu Mengdan. Study on Disease Classification of X-chest Radiographs based on Convolutional Neural Network[J].Journal of System Simulation, 2020, 32(6): 1188-1194.
5	Jacob B, Kligys S, Chen B, et al. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-only Inference[C]// IEEE Conference on Computer Vision and Pattern recognition Salt Bake City, USA: IEEE, 2018: 2704-2713.
6	Banner Ron, Nahshan Yury, Soudry Daniel. Post Training 4-bit Quantization of Convolutional Networks for Rapid-Deployment[C]// Neural Information Processing Systems(NeurIPS).Vancouver: IEEE Press,2019: 7950-7958.
7	Mishchenko Yuriy, Goren Yusuf, Ming Sun, et al. Low-Bit Quantization and Quantization-Aware Training for Small-Footprint Keyword Spotting[C]// International Conference On Machine Learning And Applications (ICMLA). Florida: IEEE Press, 2019: 706-711.
8	Zhang Xiangyu, Zhou Xinyu, Lin Mengxiao, et al.ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices [C]// Computer Vision and Pattern Recognition (CVPR). Salt Lake City, USA: IEEE Press, 2018: 6848-6856.
9	Yin Penghang, Zhang Shuai, Jiancheng Lyuet al. BinaryRelax: A Relaxation Approach for Training Deep Neural Networks with Quantized Weights[J]. SIAM Journal on Imaging Sciences (S1936-494), 2018, 11(4): 2205-2223.
10	Cao Z, Long M, Wang J, et al. Hashnet: Deep Learning to Hash by Continuation[C]// International Conference on computer vision. Venice, Italy: IEEE Press, 2017: 5608-5617.
11	Nagel Markus, van Baalen Mart, Blankevoort Tijmen, et al. Data-Free Quantization Through Weight Equalization and Bias Correction [C]// International Conference on Computer Vision (ICCV). Seoul, Korea: IEEE Press, 2019: 1325-1334.
12	Esser Steven K., McKinstry Jeffrey L., Bablani Deepika, et al. Learned Step Size Quantization[C]// International Conference on Learning Representations (ICLR).Ethiopia, Africa: IEEE Press, 2020: 1-12.
13	Bhalgat Y, Lee J, Nagel M, et al. Lsq+: Improving low-bit quantization through learnable offsets and better initialization[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle, USA: IEEE/CVF Press, 2020: 696-697.
14	Jain S, Gural A, Wu M, et al. Trained Quantization Thresholds for Accurate and Efficient Fixed-point Inference of Deep Neural Networks[J]. Machine Learning and Systems (S1002-137X), 2020, 2: 112-128.
15	Cai Zhaowei, He Xiaodong, Jian Sun, et al. Deep Learning With Low Precision by Half-Wave Gaussian Quantization[C]// Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE Press, 2017:5918-5926.
16	Choi Jungwook, Venkataramani Swagath, Vijayalakshmi (Viji) Srinivasan, et al. Accurate and Efficient 2-bit Quantized Neural Networks[J]. Proceedings of Machine Learning and Systems (S1002-137X), 2019, 1: 348-359.
17	Ioffe S, Szegedy C. Batch normalization: Accelerating Deep Network Training by Reducing Internal Covariate shift[C]// International Conference on Machine Learning. PMLR, Miami, Horida, USA: IEEE, 2015: 448-456.
18	Chmiel B, Banner R, Shomron G, et al. Robust Quantization: One Model to Rule Them All[M]. Vancouver: Advances in Neural Information Processing Systems, 2020: 5308-5317.
19	Debnath Bappaditya, O'Brien Mary, Yamaguchi Motonori, et al. Adapting MobileNets for Mobile Based Upper Body Pose Estimation[C]// Advanced Video and Signal Based Surveillance (AVSS). Auckland, Auckland: IEEE Press, 2018: 1-6.
20	Sandler M, Howard A, Zhu M, et al. Mobilenetv2: Inverted Residuals and Linear Bottlenecks[C]// IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City,USA: IEEE, 2018: 4510-4520.
21	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]// Computer Vision and Pattern Recognition. Las vegas, USA: IEEE Press, 2016: 770-778.

bit 长度	最小值	最大值
INT8	-128	127
UINT8	0	255
FP32	-3.4e38	3.4e38

序号	融合形式	融合后的算子
1	conv+BN	ConvBn算子
2	conv+BN+ReLU	ConvBnRelu算子
3	conv+BN+ReLU6	ConvBnRelu6算子

算法	量化精度（W/A）	准确率(%)Top-1	精度损失(FP32-其他)	压缩率
	MobileNet-v 2-1.0-224
FP32(ours)	32/32	72.5	-
LSQ-BN(ours)	8/8	72.3	0.2	4x
LSQ-BN(ours)	4/8	69.8	2.7	8x
PTQ	8/8	68.7	3.8	4x
PTQ	4/8	67.2	5.3	8x
QAT	8/8	70.9	1.4	4x
QAT	4/8	62	10.5	8x
LSQ	8/8	72.9	-0.4	4x
LSQ	4/8	70.3	2.2	4x

算法	量化精度（W/A）	准确率(%)Top-1	精度损失(FP32-其他)	压缩率
	ResNet-v1-50
FP32(ours)	32/32	75.6	-
LSQ-BN(ours)	8/8	75.2	0.4	4x
LSQ-BN(ours)	4/8	74.1	1.5	8x
PTQ	8/8	69.8	5.8	4x
PTQ	4/8	67.9	7.7	8x
QAT	8/8	71.2	4.4	4x
QAT	4/8	64.9	10.7	8x
LSQ	8/8	75.0	0.6	4x
LSQ	4/8	73.8	1.8	4x

算法	量化精度（W/A）	准确率(%)Top-1	精度损失(FP32-其他)	压缩率
	ResNet-v1-101
FP32(ours)	32/32	76.2	-
LSQ-BN(ours)	8/8	75.7	0.5	4x
LSQ-BN(ours)	4/8	75.1	1.1	8x
PTQ	8/8	73.6	2.6	4x
PTQ	4/8	70.4	5.8	8x
QAT	8/8	75.8	0.4	4x
QAT	4/8	74.5	1.7	8x
LSQ	8/8	75.5	0.7	4x
LSQ	4/8	75.1	1.1	4x