基于空间信息的DBN图像分类快速训练模型

系统仿真学报 ›› 2015, Vol. 27 ›› Issue (3): 549-558.

基于空间信息的DBN图像分类快速训练模型

高强, 阳武, 李倩

华北电力大学电气与电子工程学院, 保定 071003

收稿日期:2014-06-30 修回日期:2014-10-13 出版日期:2015-03-08 发布日期:2020-08-20
作者简介:高强(1960-),男,河北,博士,教授,研究方向为电力通信、深度学习以及图像处理;阳武(1975-),男,湖南,博士生,研究方向为图像处理、电力与通信系统;李倩(1990-),女,内蒙,硕士,研究方向为深度学习及图像处理。

Fast Training Model for Image Classification Based on Spatial Information of Deep Belief Network

Gao Qiang, Yang Wu, Li Qian

Institute of Electrical and Electronic Engineering, North China Electric Power University, Baoding 071003, China

Received:2014-06-30 Revised:2014-10-13 Online:2015-03-08 Published:2020-08-20

摘要/Abstract

摘要： 数据的指数级增长及算法本身的复杂性使深度信念网络(DBN)面临着学习效率问题。根据DBN的样本图像与空间信息无关的特点,建立了DBN图像分类快速训练模型,提出了基于多幅样本图像线性叠加合成思想的DBN图像分类算法—LSMI算法。利用信息熵理论,证明了样本图像与空间信息无关的特点,并以ORL库为依据进行了验证。根据正态历经性,提出了LSMI算法,并以COREL库和MIT库为仿真对象,与其他改进算法进行对比,从正确识别率和算法时间复杂度等指标,判断该算法的有效性。仿真结果表明LSMI算法在保证识别率不变的同时,大幅度降低了算法的训练时间,达到快速学习的目的。

关键词: 深度信念网络, 空间信息, 图像分类, 快速学习, LSMI算法

Abstract: With the exponential growth of data and the complexity of algorithm, efficient computing of DBN (Deep Belief Network,) has become an important issue. A fast training model for image classification of DBN was built according to sample images of DBN have nothing to do with spatial information, an improved algorithm simply as LSMI(Linear Superposition Multiple Images) was proposed for classifying images based on the idea that linear combination of multiple images. The characteristics of training images have nothing to do with spatial information was proved via information entropy theory, meanwhile, the characteristics was identified to be correct based on ORL database. According to ergodic theory, an algorithm simply as LSMI was proposed according to the ergodic theory. The LSMI algorithm was contrasted to other improved algorithms, using COREL and MIT database, judging whether the LSMI algorithm is effective from the correct recognition rate, time complexity and other quotas. The simulation results show that LSMI algorithm can ensure recognition rate, decrease training time greatly and achieve the goal of fast learning.

Key words: deep belief network, spatial information, image classification, fast learning, LSMI algorithm

中图分类号:

TP18

高强, 阳武, 李倩. 基于空间信息的DBN图像分类快速训练模型[J]. 系统仿真学报, 2015, 27(3): 549-558.

Gao Qiang, Yang Wu, Li Qian. Fast Training Model for Image Classification Based on Spatial Information of Deep Belief Network[J]. Journal of System Simulation, 2015, 27(3): 549-558.

参考文献

[1] 赵国荣, 王希彬, 高青伟. 粒子群优化小波神经网络在惯导系统传递对准中的应用[J]. 系统仿真学报, 2010, 22(3): 670-673.
[2] 董天祯, 郭江鸿, 吕娟, 等. 基于神经网络的输电线路故障检测研究[J]. 系统仿真学报, 2009, 21(15): 4903-4911.
[3] 石东源, 熊国江, 陈金福, 等. 基于径向基函数神经网络和模糊积分融合的电网分区故障诊断[J]. 中国电机工程学报, 2014, 34(4): 562-569.
[4] 侯润民, 刘荣忠, 高强, 等. 粒子群小波神经网络在交流伺服系统中的应用[J]. 系统仿真学报, 2014, 26(4): 881-886.
[5] 刘建伟, 刘媛, 罗雄麟. 玻尔兹曼机研究进展[J]. 计算机研究与发展, 2014, 51(1): 1-16.
[6] Bengio Y, Lamblin P, Popovici D, et al.Greedy Layer-wise Training of Deep Networks [C]// Advances in Neural Information Processing Systems 19 (NIPS'06). Cambridge, MA, USA: MIT Press, 2007: 153-160.
[7] Hadsell R, Chopra S, Le Cun Y.Dimensionality Reduction by Learning an Invariant Mapping[C]// IEEE Computer Society Conference on Computer Vision and Pattern Recognition (S10636919), New York, USA. USA: IEEE, 2006: 1735-1742.
[8] 赵志宏, 杨绍普, 马增强. 基于卷积神经网络LeNet-5的车牌字符识别研究[J]. 系统仿真学报, 2010, 22(3): 638-641.
[9] Schulz H, Miiller A, Behnke S.Investigating Convergence of Restricted Boltzmann Machine Learning[C]// NIPS 2010 Workshop on Deep Learning and Unsupervised Feature Learning. Canada: NISP, 2010: 1-9.
[10] 陈硕. 深度学习神经网络在语音识别中的应用研究 [D]. 广州: 华南理工大学, 2013.
[11] Ji Nannan, Zhang Jiangshe, Zhang Chunxia.A Sparse-Response Deep Belief Network Based on Rate Distortion Theory[J]. Pattern Recognition (S0031-3203), 2014, 47(9): 3179-3191.
[12] Lin Miaozhen, Xin Fan.Low Resolution Face Recognition with Pose Variations Using Deep Belief Networks[C]// 2011 4th International Congress on Image and Signal Processing. Shanghai, China: CISP, 2011: 1522-1526.
[13] Lopes N, Ribeiro B, Goncalves J.Restricted Boltzmann Machines and Deep Belief Networks on Multi-core Processors[C]// 2012 International Joint Conference on Neural Networks. Brisbane, Australia: IJCNN, 2012: 1-7.
[14] Raina R, Madhavan A, Ng A.Large-scale Deep Unsupervised Learning Using Graphics Processors[C]// Proceedings of the 26th International Conference on Machine Learning, ICML 2009. Montreal, Canada: ICML, 2009: 873-880.
[15] G E Hinton.Training Products of Experts by Minimizing Contrastive Divergence[J]. Neural Computation (S0899-7667), 2002, 14(8): 1771-1800.
[16] Cho K, Raiko T, Ilin A.Parallel Tempering is Efficient for Learning Restricted Boltzmann Machines[C]// 2010 International Joint Conference on Neural Networks. New York, USA: IJCNN, 2010: 1-8.
[17] Tieleman T.Training Restricted Boltzmann Machines Using Approximations to the Likelihood gradient[C]// Proceedings of the 25th international conference on Machine learning. Canada: ICML, 2008: 1064-1071.
[18] Cho K, Raiko T, Ilin A.Enhanced Gradient and Adaptive Learning Rate for Training Restricted Boltzmann Machines[C]// Proceedings of the 28th International Conference on Machine Learning. New York, USA: ICML, 2011: 105-112.
[19] Lopes N, Ribeiro B.Improving Convergence of Restricted Boltzmann Machines via a Learning Adaptive Step Size[C]// Proceedings off the 17th Iberoamerican Congress on Pattern Recognition CIARP 2012. Berlin, Germany: Springer, 2012: 511-518.
[20] Lopes N, Ribeiro B.Towards Adaptive Learning with Improved Convergence of Deep Belief Networks on Graphics Processing Units[J]. Pattern Recognition (S0031-3203), 2014, 47(1): 114-127.
[21] Martens J.Deep Learning Via Hessian-free Optimization[C]// Proceedings of the 27th International on Machine Learning, ICML 2010. Haifa, Israel: ICML, 2010: 735-742.
[22] 郑志蕴, 李步源, 李伦, 等. 基于云计算的受限玻尔兹曼机推荐算法研究[J]. 计算机科学, 2013, 40(12): 259-263.
[23] 吴川. 基于神经网络的目标识别及定位方法的研究 [D]. 长春: 中国科学院长春光学精密机械与物理研究所, 2005.