基于曲率信息的改进BP算法及其在FNN中的应用

doi:10.16182/j.issn1004731x.joss.17-0461

系统仿真学报 ›› 2020, Vol. 32 ›› Issue (1): 1-8.doi: 10.16182/j.issn1004731x.joss.17-0461

• 仿真建模理论与方法 • 下一篇

基于曲率信息的改进BP算法及其在FNN中的应用

熊伟丽^1,2, 孙文心¹, 史旭东¹

1. 江南大学物联网工程学院,江苏无锡 214122;
2. 江南大学轻工过程先进控制教育部重点实验室,江苏无锡 214122

收稿日期:2017-11-08 修回日期:2018-01-18 发布日期:2020-01-17
作者简介:熊伟丽(1978-),女,河南洛阳,博士,教授,硕导,研究方向为复杂工业过程建模及优化,智能优化算法及应用;孙文心(1993-),男,江苏无锡,硕士生,研究方向为复杂工业过程建模及优化。
基金资助:
国家自然科学基金(61773182)

Curvature-based BP Algorithm Optimization and Its Application in FNN

Xiong Weili^1,2, Sun Wenxin¹, Shi Xudong¹

1. School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, China;
2. Key Laboratory of Advanced Process Control for Light Industry (Ministry of Education), Jiangnan University, Wuxi 214122, China

Received:2017-11-08 Revised:2018-01-18 Published:2020-01-17

摘要/Abstract

摘要： 针对步长选取影响误差反向传播(BP,Back Propagation)算法优化效率问题,提出一种基于曲率信息的步长优化BP算法,并将其应用到了模糊神经网络(FNN)的训练过程中。参考牛顿法的思想,根据代价函数的梯度及梯度方向上的曲率信息来确定模型参数调整的方向和幅度。仅需考虑梯度方向上的二阶信息,因此不需要存储和处理Hessian矩阵。通过一个数值仿真和高炉炼铁过程数据建模实验,验证了方法的有效性及训练效率。

关键词: 步长优化, BP算法, 模糊神经网络, 学习速度

Abstract: In order to improve the optimization efficiency of BP algorithm affected by the selection of step size, a step size optimization BP algorithm based on curvature information is proposed and applied to the training process of FNN (Fuzzy Neural Network). Reference to Newton's method, The gradient of the cost function and the curvature information in the direction are calculated to determine the direction and magnitude of the parameter adjustment in each iteration. This method only needs to consider the two order information of the gradient direction, so it does not need the storage and processing of Hessian matrix. The effectiveness and efficiency of the proposed method are verified by a numerical simulation and data simulation of blast furnace ironmaking process.

Key words: Step size optimization, BP algorithm, Fuzzy neural network, Learning speed

中图分类号:

TP183

熊伟丽, 孙文心, 史旭东. 基于曲率信息的改进BP算法及其在FNN中的应用[J]. 系统仿真学报, 2020, 32(1): 1-8.

Xiong Weili, Sun Wenxin, Shi Xudong. Curvature-based BP Algorithm Optimization and Its Application in FNN[J]. Journal of System Simulation, 2020, 32(1): 1-8.

参考文献

[1] 王海军. 带自适应步长神经网络学习算法的收敛性分析[D]. 大连: 大连理工大学, 2016.
Wang Haijun.Convergence Analysis of Neural Network Learning Algorithm with Adaptive Learning Rate[D]. Dalian: Dalian University of Technology, 2016.
[2] LeCun Y A, Bottou L, Orr G B, et al. Efficient backprop[M]. Neural networks: Tricks of the trade. Springer Berlin Heidelberg, 2012: 9-48.
[3] Nesterov Y.Gradient methods for minimizing composite functions[J]. Mathematical Programming (S0025-5610), 2013, 140(1): 125-161.
[4] 房振勇, 游文虎, 冯汝鹏. 改进BP算法在模糊神经网络中的应用[J]. 北京航空航天大学学报, 2007, 33(11): 1321-1324.
Fang Zhenyong, You Wenhu, Feng Rupeng.Application of improved BP algorithm in fuzzy neural networks[J]. Journal of Beijing University of Aeronautics and Astronautics, 2007, 33(11): 1321-1324.
[5] 高雪鹏, 丛爽. BP网络改进算法的性能对比研究[J]. 控制与决策, 2001, 16(2): 167-171.
Gao Xuepeng, Cong Shuang.Comparative Study on Fast Learning Algorithms of BP Networks[J]. Control and Decision, 2001, 16(2): 167-171.
[6] Hadgu A T, Nigam A, Diaz-Aviles E.Large-scale learning with AdaGrad on Spark[C]// IEEE International Conference on Big Data. Beijing, China: IEEE, 2015: 2828-2830.
[7] Goodfellow I, Bengio Y, Courville A.Deep learning[M]. Massachusetts, USA: MIT Press, 2016.
[8] Mukkamala M C, Hein M.Variants of RMSProp and Adagrad with logarithmic regret bounds[C]// Proceedings of the 34th International Conference on Machine Learning, PMLR 70. Sydney, Australia: ICML, 2017: 2545-2553.
[9] Wang L X.A course in fuzzy systems and control[M]. Englewood, USA: Prentice-Hall, Inc. 1996.
[10] Hecht-Nielsen.Theory of the backpropagation neural network[C]// International Joint Conference on Neural Networks. Honolulu, HI, USA: IEEE, 2002(1): 593-605.
[11] 董海鹰, 魏占宏, 杨玺, 等. 基于自适应模糊神经网络的风速软测量[J]. 电力系统及其自动化学报, 2013, 25(1): 60-65.
Dong Haiying, Wei Zhanhong, Yang Xi, et al.Wind Speed Soft Sensor Based on Adaptive Fuzzy Neural Network[J]. Proceedings of the CSU-EPSA, 2013, 25(1): 60-65.
[12] Pratama M, Er M J, Li X, et al.Data driven modeling based on dynamic parsimonious fuzzy neural network[J]. Neurocomputing (S0925-2312), 2013, 110: 18-28.
[13] Zhang M, Liu X.A soft sensor based on adaptive fuzzy neural network and support vector regression for industrial melt index prediction[J]. Chemometrics & Intelligent Laboratory Systems (S0169-7439), 2013, 126: 83-90.
[14] 李国勇, 杨丽娟. 神经·模糊·预测控制及其MATLAB实现. [M]. 3版. 北京: 电子工业出版社, 2013: 257-260.
Li Guoyong, Yang Lijuan. Neural Netowrk.Fuzzy. Predictive Control and its MATLAB implementation [M]. 3 Edition. Beijing: Publishing House of Electronics Industry, 2013: 257-260.
[15] 丛爽. 几种模糊神经网络系统关系的对比研究[J]. 信息与控制, 2001, 30(6): 486-491.
Cong Shuang.Comparative Research on Relationships between Several Fuzzy-Neural Network Systems[J]. Information and Control, 2001, 30(6): 486-491.
[16] 李炯城, 肖恒辉, 李桂愉. 高效的线搜索寻优方法[J]. 计算机工程与应用, 2013, 49(3): 53-56.
Li Jiongcheng, Xiao Henghui, Li Guiyu.Efficient line search optimization method[J]. Computer Engineering and Applications, 2013, 49(3): 53-56.
[17] 蒋朝辉, 董梦林, 桂卫华, 等. 基于Bootstrap的高炉铁水硅含量二维预报[J]. 自动化学报, 2015, 42(5): 715-723.
Jiang Zhaohui, Dong Menglin, Gui Weihua, et al.Two-dimensional Prediction for Silicon Content of Hot Metal of Blast Furnace Based on Bootstrap[J]. Acta Automatica Sinica, 2015, 42(5): 715-723.
[18] Mika S, Lkopf B, Smola A, et al.Kernel PCA and de-noising in feature spaces[C]// Conference on Advances in Neural Information Processing Systems. Marriott, Denver, USA: MIT Press, 1999: 536-542.

基于曲率信息的改进BP算法及其在FNN中的应用

Curvature-based BP Algorithm Optimization and Its Application in FNN

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

本文评价

[1]	李霏, 苏中. 基于自适应免疫优化的污水处理过程控制[J]. 系统仿真学报, 2021, 33(12): 3012-3020.
[2]	杨祯山, 岳文姣. 基于FCM聚类模糊神经网络的电梯交通模式识别[J]. 系统仿真学报, 2018, 30(4): 1433-1439.
[3]	王娟, 杨小渝, 王宗国, 任杰, 赵旭山. 基于混合学习算法的材料计算数据误差估计[J]. 系统仿真学报, 2018, 30(10): 3843-3853.