Evolutionary Extreme Learning Machine Optimized by Quantum-behaved Particle Swarm Optimization

doi:10.16182/j.issn1004731x.joss.201710028

Abstract

Abstract: Extreme Learning Machine (ELM) is a novel learning algorithm for Single-Hidden-Layer Feed Forward Neural Networks (SLFN) with much faster learning speed and better generalization performance than traditional gradient-based learning algorithms. However ELM tends to require more neurons in the hidden layer and lead to ill-conditioned problem due to the random selection of input weights and hidden biases. To address these problems, a learning algorithm was proposed which used quantum-behaved particle swarm optimization (QPSO) to select the optimal network parameters including the number of hidden layer neurons according to the both the root mean square error on validation data set and the norm of output weights. Experimental results on benchmark regression and classification problems have verified the performance and effectiveness of the proposed approach.

Key words: extreme learning machine, single-hidden-layer feed forward neural networks, quantum-behaved particle swarm optimization, generalization performance

CLC Number:

TP183

Pang Shan, Yang Xinyi, Lin Xuesen. Evolutionary Extreme Learning Machine Optimized by Quantum-behaved Particle Swarm Optimization[J]. Journal of System Simulation, 2017, 29(10): 2447-2458.

References

[1] G B Huang, Q Y Zhu, C K Siew.Extreme Learning Machine: a new learning scheme of feed forward neural networks[C]// 2004 International Joint Conference on Neural Networks, Budapest, Hungary. Piscataway, NJ, USA: IEEE, 2004: 985-990.
[2] G B Huang, Q Y Zhu, C K Siew.Extreme learning machine: theory and applications[J]. Neurocomputing (S0925-2312), 2006, 70(1/3): 489-501. doi:10.1016/j. neucom. 2005.12.126.
[3] Q Y Zhu, A K Qin, P N Suganthan, et al.Evolutionary extreme learning machine[J]. Pattern Recognition (S0031-3203), 2005, 38(10): 1759-1763. doi: 10.1016/j. patcog. 2005.03.028.
[4] Cao Z Lin, G B Huang. Self-adaptive evolutionary extreme learning machine[J]. Neural Processing Letter (S 1370-4621), 2012, 36(3): 285-305. doi:10.1007/s11063- 012-9236-y.
[5] Tiago Matias, Francisco Souza, Rui Araújo, et al.Learning of a single-hidden layer feed forward neural network using an optimized extreme learning machine[J]. Neurocomputing (S0925-2312), 2014, 129: 428-436, doi: 10.1016/j.neucom.2013.09.016.
[6] C Chang-Yi, Y Yong-Chun.Solving of nonlinear equations based on PSO algorithm[J]. Computer Application and Software (S1000-386x), 2006, 23(5): 137-139.
[7] Z Miao, S Xie, Y Wu, et al.Aero-engine state variable modeling based on the improved particle swarm optimization[J]. Journal of Propulsion Technology (S 1001-4055), 2012, 33(1): 73-77.
[8] A H Gandomi, G J Yun, X-S Yang, et al.Chaos-enhanced accelerated particle swarm optimization[J]. Communications in Nonlinear Science and Numerical Simulation (S1007-5704), 2013, 18(2): 327-340, doi:10.1016/j. cnsns. 2012.07.017
[9] Y Xu, Y Shu.Evolutionary extreme learning machine based on particle swarm optimization[J]. Lecture Notes in Computer Science (S0302-9743), 2006, 36(3): 644-652.
[10] F Han, H Yao, Q Ling.An improved extreme learning machine based on particle swarm optimization[J]. Neurocomputing (S0925-2312), 2013, 116: 87-93, doi: 10.1016/j.neucom.2011.12.062.
[11] F vanden Bergh, A P Engelbrecht. A study of particle swarm optimization particle trajectories[J]. Information Sciences (S 0020-0255), 2006, 176(8): 937-971.
[12] J Sun, C H Lai, W B Xu, et al.A modified quantum-behaved particle swarm optimization[C]// Proceedings of the 7th International Conference on Computational Science (ICCS '07), Beijing, China. Germany: Springer, 2007: 294-301.
[13] Li R, Li W J, Zhang L, et al.An improved quantum-behaved particle swarm classifier based on weighted mean best position[C]// 2009 IEEE International Conference on Intelligent Computing and Intelligent Systems, Shanghai, China. USA: IEEE, 2009:327-331.
[14] Jianhua Xiao, Jin Xu, Zhihua Chen.A hybrid quantum chaotic swarm evolutionary algorithm for DNA encoding[J]. Computers and Mathematics with Applications (S0898-1221), 2009, 57(11-12): 1949-1958.
[15] D Chen, J Wang, F Zou, et al.An improved group search optimizer with operation of quantum-behaved swarm and its application[J]. Applied Soft Computing Journal (S 1568-4946), 2012, 2(2): 712-725.
[16] J Sun, W Fang, V Palade et al. Quantum-behaved particle swarm optimization with Gaussian distributed local attractor point[J]. Applied Mathematics and Computation (S 0096-3003), 2011, 218(7): 3763-3775.
[17] R C Eberhart, J Kennedy.A new optimizer using particles swarm theory[C]// Proceedings of the Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, 1995. USA: IEEE, 1995: 39-43.
[18] R C Eberhart, J Kennedy.Particle swarm optimization.[C]// Proceeding of the IEEE International Conference on Neural Network, Perth, Australia, 1995. USA: IEEE, 1995: 1942-1948.
[19] G B Huang. Sinc.zip [DB/OL]. (2004-04-16) [2015-07-10].http://www.ntu.edu.sg/home/egbhuang/elm_codes.html.