Fast Training Model for Image Classification Based on Spatial Information of Deep Belief Network

Abstract

Abstract: With the exponential growth of data and the complexity of algorithm, efficient computing of DBN (Deep Belief Network,) has become an important issue. A fast training model for image classification of DBN was built according to sample images of DBN have nothing to do with spatial information, an improved algorithm simply as LSMI(Linear Superposition Multiple Images) was proposed for classifying images based on the idea that linear combination of multiple images. The characteristics of training images have nothing to do with spatial information was proved via information entropy theory, meanwhile, the characteristics was identified to be correct based on ORL database. According to ergodic theory, an algorithm simply as LSMI was proposed according to the ergodic theory. The LSMI algorithm was contrasted to other improved algorithms, using COREL and MIT database, judging whether the LSMI algorithm is effective from the correct recognition rate, time complexity and other quotas. The simulation results show that LSMI algorithm can ensure recognition rate, decrease training time greatly and achieve the goal of fast learning.

Key words: deep belief network, spatial information, image classification, fast learning, LSMI algorithm

CLC Number:

TP18

Gao Qiang, Yang Wu, Li Qian. Fast Training Model for Image Classification Based on Spatial Information of Deep Belief Network[J]. Journal of System Simulation, 2015, 27(3): 549-558.

References

[1] 赵国荣, 王希彬, 高青伟. 粒子群优化小波神经网络在惯导系统传递对准中的应用[J]. 系统仿真学报, 2010, 22(3): 670-673.
[2] 董天祯, 郭江鸿, 吕娟, 等. 基于神经网络的输电线路故障检测研究[J]. 系统仿真学报, 2009, 21(15): 4903-4911.
[3] 石东源, 熊国江, 陈金福, 等. 基于径向基函数神经网络和模糊积分融合的电网分区故障诊断[J]. 中国电机工程学报, 2014, 34(4): 562-569.
[4] 侯润民, 刘荣忠, 高强, 等. 粒子群小波神经网络在交流伺服系统中的应用[J]. 系统仿真学报, 2014, 26(4): 881-886.
[5] 刘建伟, 刘媛, 罗雄麟. 玻尔兹曼机研究进展[J]. 计算机研究与发展, 2014, 51(1): 1-16.
[6] Bengio Y, Lamblin P, Popovici D, et al.Greedy Layer-wise Training of Deep Networks [C]// Advances in Neural Information Processing Systems 19 (NIPS'06). Cambridge, MA, USA: MIT Press, 2007: 153-160.
[7] Hadsell R, Chopra S, Le Cun Y.Dimensionality Reduction by Learning an Invariant Mapping[C]// IEEE Computer Society Conference on Computer Vision and Pattern Recognition (S10636919), New York, USA. USA: IEEE, 2006: 1735-1742.
[8] 赵志宏, 杨绍普, 马增强. 基于卷积神经网络LeNet-5的车牌字符识别研究[J]. 系统仿真学报, 2010, 22(3): 638-641.
[9] Schulz H, Miiller A, Behnke S.Investigating Convergence of Restricted Boltzmann Machine Learning[C]// NIPS 2010 Workshop on Deep Learning and Unsupervised Feature Learning. Canada: NISP, 2010: 1-9.
[10] 陈硕. 深度学习神经网络在语音识别中的应用研究 [D]. 广州: 华南理工大学, 2013.
[11] Ji Nannan, Zhang Jiangshe, Zhang Chunxia.A Sparse-Response Deep Belief Network Based on Rate Distortion Theory[J]. Pattern Recognition (S0031-3203), 2014, 47(9): 3179-3191.
[12] Lin Miaozhen, Xin Fan.Low Resolution Face Recognition with Pose Variations Using Deep Belief Networks[C]// 2011 4th International Congress on Image and Signal Processing. Shanghai, China: CISP, 2011: 1522-1526.
[13] Lopes N, Ribeiro B, Goncalves J.Restricted Boltzmann Machines and Deep Belief Networks on Multi-core Processors[C]// 2012 International Joint Conference on Neural Networks. Brisbane, Australia: IJCNN, 2012: 1-7.
[14] Raina R, Madhavan A, Ng A.Large-scale Deep Unsupervised Learning Using Graphics Processors[C]// Proceedings of the 26th International Conference on Machine Learning, ICML 2009. Montreal, Canada: ICML, 2009: 873-880.
[15] G E Hinton.Training Products of Experts by Minimizing Contrastive Divergence[J]. Neural Computation (S0899-7667), 2002, 14(8): 1771-1800.
[16] Cho K, Raiko T, Ilin A.Parallel Tempering is Efficient for Learning Restricted Boltzmann Machines[C]// 2010 International Joint Conference on Neural Networks. New York, USA: IJCNN, 2010: 1-8.
[17] Tieleman T.Training Restricted Boltzmann Machines Using Approximations to the Likelihood gradient[C]// Proceedings of the 25th international conference on Machine learning. Canada: ICML, 2008: 1064-1071.
[18] Cho K, Raiko T, Ilin A.Enhanced Gradient and Adaptive Learning Rate for Training Restricted Boltzmann Machines[C]// Proceedings of the 28th International Conference on Machine Learning. New York, USA: ICML, 2011: 105-112.
[19] Lopes N, Ribeiro B.Improving Convergence of Restricted Boltzmann Machines via a Learning Adaptive Step Size[C]// Proceedings off the 17th Iberoamerican Congress on Pattern Recognition CIARP 2012. Berlin, Germany: Springer, 2012: 511-518.
[20] Lopes N, Ribeiro B.Towards Adaptive Learning with Improved Convergence of Deep Belief Networks on Graphics Processing Units[J]. Pattern Recognition (S0031-3203), 2014, 47(1): 114-127.
[21] Martens J.Deep Learning Via Hessian-free Optimization[C]// Proceedings of the 27th International on Machine Learning, ICML 2010. Haifa, Israel: ICML, 2010: 735-742.
[22] 郑志蕴, 李步源, 李伦, 等. 基于云计算的受限玻尔兹曼机推荐算法研究[J]. 计算机科学, 2013, 40(12): 259-263.
[23] 吴川. 基于神经网络的目标识别及定位方法的研究 [D]. 长春: 中国科学院长春光学精密机械与物理研究所, 2005.