基于霍普菲尔德网络的云作业调度算法

doi:10.16182/j.issn1004731x.joss.19-FZ0323

系统仿真学报 ›› 2019, Vol. 31 ›› Issue (12): 2859-2867.doi: 10.16182/j.issn1004731x.joss.19-FZ0323

基于霍普菲尔德网络的云作业调度算法

郭玉栋¹, 左金平^2,*

1. 晋中学院网络信息中心,山西晋中 030600;
2. 晋中学院信息技术与工程学院,山西晋中 030600

收稿日期:2019-05-05 修回日期:2019-07-15 发布日期:2019-12-13
作者简介:郭玉栋(1974-),男,山西太谷,硕士,副教授,研究方向为大数据及云计算技术; 左金平(通讯作者1975-),女,山西长治,硕士,副教授,研究方向为大数据及云计算技术。
基金资助:
山西省高等学校科技创新基金(20171118),山西省软科学计划研究基金(2016041008-6), 山西省高等学校教学改革创新项目(J2019183)

The Scheduling Algorithm of Cloud Job Based on Hopfield Neural Network

Guo Yudong¹, Zuo Jinping^2,*

1. Network Information Center, Jinzhong University, JinZhong 030600, China;
2. School of Information Technology & Engineering , Jinzhong University, JinZhong 030600, China;

Received:2019-05-05 Revised:2019-07-15 Published:2019-12-13

摘要/Abstract

摘要： 针对当前云作业调度效率不高,资源利用不够充分,尚不能发挥其最大优势,提出一种基于霍普菲尔德网络的作业调度算法。为了实现系统资源调度能力的提高,分析影响云作业调度相关资源的特点;建立资源条件约束数学模型,再设计霍普菲儿德能量函数,并对其优化;通过标准用例集进行测试分析9个节点的平均利用率,并与3个典型算法进行性能和资源利用方面的比较。实验表明,该方法在效率上较其它3个算法有显著提升。

关键词: Hadoop, 云调度算法, Hopfield neural network, MapReduce, 优化算法

Abstract: Focusing on the low efficiency of cloud job scheduling and the insufficient utility of resource, a job scheduling algorithm based on Hopfield Neural Network is proposed. In order to improve the resource scheduling ability of the system, The resource characteristics which influence the cloud job scheduling are shown. The mathematical model of resource constraints is established, and the Hopfield energy function is designed and optimized. The average utilization rate of 9 nodes is analyzed by using the standard test cases, and the performance and resource utilization of the proposed strategy are compared with three typical algorithms. The results show that the average efficiency of the cloud job scheduling based on the algorithm is improved significantly.

Key words: Hadoop, cloud scheduling algorithm, Hopfield neural network, MapReduce, optimization algorithm

中图分类号:

TP391.9

郭玉栋, 左金平. 基于霍普菲尔德网络的云作业调度算法[J]. 系统仿真学报, 2019, 31(12): 2859-2867.

Guo Yudong, Zuo Jinping. The Scheduling Algorithm of Cloud Job Based on Hopfield Neural Network[J]. Journal of System Simulation, 2019, 31(12): 2859-2867.

参考文献

[1] Vaibhav P, Poonam S.How Heterogeneity Affects the Design of Hadoop MapReduce Schedulers: A State-of-the-Art Survey and Challenges[J]. Big Data (S2167-6461), 2018, 6(2): 72-95.
[2] Jafarnejad G E, Masoud R A, Nasih Q N.Load-balancing algorithms in cloud computing: A survey[J]. Journal of Network and Computer Applications (S1084- 8045), 2017, 88: 50-71.
[3] Nima K S, Jafari N N.MapReduce and its Applications, Challenges, and Architecture: a Comprehensive Review and Directions for Future Research[J]. Journal of Grid Computing (S1570-7873), 2017, 15(3): 295-321.
[4] Shivnath B.Towards automatic optimization of MapReduce programs[C]. Proceedings of the 1st ACM Symposium on Cloud Computing (SoCC 10), Indianapolis, USA, Jun 10- 11, 2010. New York, NY, USA: ACM, 2010: 137-142.
[5] Herodotou H, Lim H, Luo Gang, et al.Starfish: a self-tuning system for big data analytics[C]. Proceedings of the 5th Biennial Conference on Innovative Data Systems Research (CIDR 11), Asilomar, USA, Jan 9-12. 2011: 261-272.
[6] Herodotos H, Shivnath B.Profiling, what- if analysis, and cost-based optimization of MapReduce programs[C]. Proceedings of the 36th International Conference on Very Large Data Bases (VLDB 10), Singapore, Sep 13-17. 2010: 1111-1122.
[7] Tian C, Zhou H J, He Y Q, et al.A dynamic MapReduce scheduler for heterogeneous workloads[C]. Proceedings of the 8th International Conference on Grid and Cooperative Computing (GCC 09), Aug 27- 29, Lanzhou, China. 2009: 218-224.
[8] Jahani E, Cafarella M J, Re C.Automatic optimization for MapReduce programs[C]. Proceedings of the 37th International Conference on Very Large Data Bases (VLDB 11), Seattle, USA, Aug 29-Sep 3.2011: 385-396.
[9] Ahmad F, Chakradhar S T, Raghunathan A, et al.Tarazu: optimizing MapReduce on heterogeneous clusters[C]. Proceedings of the 7th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 12), London, UK, May 3-7. 2012: 61-74.
[10] Polo J, Carrera D, Becerra Y, et al.Performance management of accelerated MapReduce workloads in heterogeneous clusters[C]. Proceedings of the 2010 39^th International Conference on Parallel Processing (ICPP 10), San Diego, USA, Sep 13- 16, 2010. Washington, DC. USA: IEEE Computer Society, 2010: 653-662.
[11] Fadika Z, Dede E, Hartog J, et al.MARLA: MapReduce for heterogeneous clusters[C]. Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 12), Ottawa, Canada, May 13-16, 2012. Washington, DC. USA: IEEE Computer Society, 2012: 49-56.
[12] Zhuo T, Zhou J Q, Li K L, Li R X.A MapReduce task scheduling algorithm for deadline constraints[J]. Cluster Computing (S1386-7857), 2013, 16(4): 651-662.
[13] 马莉, 唐善成, 王静, 等. 云计算环境下的动态反馈作业调度算法[J]. 西安交通大学学报, 2014, 48(7): 77-82.
Ma Li, Tang Shancheng, Wang Jing, et al.Dynamic Feedback Job Scheduling Algorithm in Cloud Computing[J]. Journal of Xi’an Jiaotong University, 2014, 48(7): 77-82.
[14] Xu X Y, Tang M L, Tian Y C.QoS-guaranteed Resource Provisioning for Cloud-Based MapReduce in Dynamical Environments[J]. Future Generation Computer System (S0167-739X), 2017, 78: 18-30.
[15] Suresh S, Gopalan N P.An optimal task selection scheme for Hadoop scheduling[J]. IERI Procedia (S2212-6678), 2014, 10: 70-75.
[16] Rasooli A, Down D G, COSHH: A classification and optimization based scheduler for heterogeneous Hadoop systems[J]. Future Generation computer Systems (S0167-739X), 2014, (36): 1-15.
[17] Zhang F, Cao J, Li K, et al.Multi-objective scheduling of many tasks in cloud platforms[J]. Future Generation Computer Systems (S0167-739X), 2014, 37: 309-320.
[18] 夏家莉, 陈辉, 杨兵. 一种动态优先级实时任务调度算法[J]. 计算机学报, 2012, 35(12): 2685-2695.
Xia Jiali, Chen Hui, Yang Bin.A Real-Time Tasks Scheduling Algorithm Based on Dynamic Priority[J]. Chinese Journal of Computers, 2012, 35(12): 2685-2695.
[19] 宋杰, 徐澍, 郭朝鹏, 等. 一种优化MapReduce系统能耗的任务分发算法[J]. 计算机学报, 2016, 39(2): 323-338.
Song Jie, Xu Shu, Guo Chaopeng, et al.A Task Distribution Algorithm for Energy Consumption Optimization of Map Reduce System[J]. Chinese Journal of computers, 2016, 39(2): 323-338.
[20] Hopfield J J, Tank D W.Neural computation of decision in optimization problems[J]. Biol. Cybern (S0340-1200), 1985, 52: 141-152.
[21] 王万良, 吴启迪, 徐新黎. 基于Hopfield神经网络的作业车间生产调度方法[J]. 自动化学报, 2002, 28(5): 838-844.
Wang Wanliang, Wu Qidi, Xu Xinli.Hopfield Neural Network Approach for Job-Shop Scheduling Problems[J]. Acta Automatica Sinica, 2002, 28(5): 838-844.
[22] 王秀利, 吴惕华. 一种求解多处理器作业调度的Hopfield神经网络方法[J]. 系统工程与电子技术, 2002, 24(8): 13-16.
Wang Xiuli, Wu Tihua.Scheduling Mutiprocessor Job Using Hopfield Neural Network[J]. Systems Engineering and Electronics, 2002, 24(8): 13-16.
[23] 徐昇, 业宁, 朱发, 等. Hopfield网络在视差空间上的立体匹配求解[J]. 计算机研究与发展, 2013, 50(5): 1021-1029.
Xu Sheng, Ye Ning, Zhu Fa, et al.Solving the Stereo Matching in Disparity Image Space Using the Hopfield Neural Network[J]. Journal of Computer Research and Development, 2013, 50(5): 1021-1029.

基于霍普菲尔德网络的云作业调度算法

The Scheduling Algorithm of Cloud Job Based on Hopfield Neural Network

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张其文, 张斌. 基于教学优化算法求解置换流水车间调度问题[J]. 系统仿真学报, 2022, 34(5): 1054-1063.
[2]	高鑫宇, 倪静. 救援效率视角下灾后动态应急配送网络优化[J]. 系统仿真学报, 2022, 34(4): 806-816.
[3]	李建锋, 卢迪, 李贺香. 一种改进的原子搜索算法[J]. 系统仿真学报, 2022, 34(3): 490-502.
[4]	党建武, 谭凌. 改进果蝇算法优化加权极限学习机的入侵检测[J]. 系统仿真学报, 2021, 33(2): 331-338.
[5]	高昂, 王晓路, 董志明, 张国辉. 装甲兵虚拟兵力战术对抗动作参数矫正方法[J]. 系统仿真学报, 2021, 33(2): 484-493.
[6]	王宁, 魏利胜. 一种基于GA的新型生物地理学优化算法研究[J]. 系统仿真学报, 2020, 32(9): 1717-1723.
[7]	王闯, 张勇, 李学贵, 董宏丽. 改进粒子群优化算法及其在聚类分析中应用[J]. 系统仿真学报, 2020, 32(8): 1577-1587.
[8]	刘长平, 简祯富, 傅文翰. 求解零等待流水线调度问题的离散磷虾群算法[J]. 系统仿真学报, 2020, 32(6): 1051-1059.
[9]	黄光球, 陆秋琴. 人感禽流感优化算法[J]. 系统仿真学报, 2020, 32(5): 767-781.
[10]	於万里, 王艳, 纪志成. 氨糖发酵过程建模与工艺参数优化研究[J]. 系统仿真学报, 2020, 32(10): 1895-1902.
[11]	陆湛文, 程新功, 张永峰. 基于共识粒子群的全局优化求解方法[J]. 系统仿真学报, 2020, 32(10): 1936-1942.
[12]	陈海亮, 陈彬, 袁鹏, 董健, 艾川. 基于观测数据的社交网络推断方法研究[J]. 系统仿真学报, 2019, 31(12): 2712-2720.
[13]	李荣雨, 梁栋, 戚桂洪. 基于自适应反馈机制的精英教学优化算法[J]. 系统仿真学报, 2018, 30(8): 2950-2957.
[14]	刘长良, 王鹏飞, 刘帅, 罗磊, 回振桥. 一种改进的CS算法及其在微电网优化中的应用[J]. 系统仿真学报, 2018, 30(3): 930-936.
[15]	曾艳阳, 张泽浩, 冯云霞. 自适应可调混沌粒子群算法的体绘制视点选择[J]. 系统仿真学报, 2018, 30(12): 4595-4601.