绿色数据中心数据处理型框架中的数据管理

系统仿真学报 ›› 2016, Vol. 28 ›› Issue (3): 592-599.

绿色数据中心数据处理型框架中的数据管理

张啸¹, 高原², 王晓亮¹, 葛以踊², 杨海祥¹, 万书鹏²

1.南京大学计算机软件新技术国家重点实验室南京大学计算机科学与技术系,南京 210023;
2.国电南瑞科技股份有限公司,南京 210023

收稿日期:2014-07-15 修回日期:2014-10-24 发布日期:2020-07-02
作者简介:张啸(1991-),男,山东,硕士生,研究方向为数据中心。
基金资助:
国家自然科学基金(61370028, 91218302, 61321491);江苏省自然科学基金(BK2011191);江苏省科技支撑计划(BE2013116);中央高校基本科研业务费专项资金(20620140514);国家电网公司科技项目

Data Management of Data Processing Framework in Green Data Center

Zhang Xiao¹, Gao Yuan², Wang Xiaoliang¹, Ge Yiyong², Yang Haixiang¹, Wan Shupeng²

1. State Key Laboratory for Novel Software Technology, Department of Computer Science and Technology, Nanjing University, Nanjing 210023, China
2. NARI Technology Co, Ltd, Nanjing 210023, China

Received:2014-07-15 Revised:2014-10-24 Published:2020-07-02

摘要/Abstract

摘要： 使用绿色能源已成为解决数据中心能耗问题的一种有效方式。为了降低绿色能源变化幅度大的特点带来的影响,通常将可延迟作业放入等待队列,将相应空闲服务器置为休眠状态,降低系统能耗,在新能源可用的时候执行作业。当新作业执行时,需要重新开启休眠状态服务器来保证数据可用性。数据放置与作业执行时间的不统一,会导致频繁开启休眠服务器,带来能源浪费。针对绿色数据中心提出一种数据调度策略,根据数据处理型框架中等待队列作业调度次序,通过将未来一段时间内需要被读取的数据块提前复制在活跃服务器上,降低休眠状态服务器开启的次数,从而降低总体能耗。实验模拟结果显示, 该算法可平均减少43%的休眠状态服务器重复开启次数。

关键词: 绿色数据中心, 数据处理型框架, 能耗, 新能源, 数据管理

Abstract: Using renewable energy in data center is an environment-friendly way to solve the problem of high energy consumption of data center. Since renewable energy is variable, delaying the jobs which has no strict deadline i a widely used strategy to maximize the usage of renewable energy. Meanwhile, turning the idle servers off can further reduce energy consumption. If the data required by the jobs to be processed are unavailable, some servers in sleep state need to be reactivated to guarantee that the data required by the jobs are available. Such operation may lead to energy waste due to the frequent reactive processes. An effective data management algorithm was proposed, which copied the data required by the jobs in waiting queue to active servers in advance. By doing so, the times that the sleep servers were reactivated could be reduced. Simulation results show that the times can be reduced by 43% on average.

Key words: green data center, data processing framework, energy, renewable energy, data management

中图分类号:

TP391.9

张啸, 高原, 王晓亮, 葛以踊, 杨海祥, 万书鹏. 绿色数据中心数据处理型框架中的数据管理[J]. 系统仿真学报, 2016, 28(3): 592-599.

Zhang Xiao, Gao Yuan, Wang Xiaoliang, Ge Yiyong, Yang Haixiang, Wan Shupeng. Data Management of Data Processing Framework in Green Data Center[J]. Journal of System Simulation, 2016, 28(3): 592-599.

参考文献

[1] Gao P X, Curtis A P, Wong B, Keshav S.It’s not easy being green[C]// Proceedings of the ACM SIGCOMM 2012, Helsinki, Finland. USA: ACM, 2012: 211-222.
[2] Deng W, Liu F, Jin H, et al.Harnessing renewable energy in cloud datacenters: opportunities and challenges[J]. IEEE Network (S0890-8044), 2014, 28(1): 48-55.
[3] Goiri Í, Le K, Nguyen T D, et al.GreenHadoop: leveraging green energy in data-processing frameworks[C]// Proceedings of the 7th ACM European Conference on Computer Systems. USA: ACM, 2012: 57-70.
[4] Goiri Í, Beauchea R, Le K, et al.GreenSlot: scheduling energy consumption in green datacenters[C]// Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis. USA: ACM, 2011: 20.
[5] Goiri Í, Katsak W, Le K, et al.Parasol and GreenSwitch: Managing datacenters powered by renewable energy[C]// Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems. USA: ACM, 2013: 51-64.
[6] Aksanli B, Venkatesh J, Zhang L, et al.Utilizing green energy prediction to schedule mixed batch and service jobs in data centers[J]. ACM SIGOPS Operating Systems Review (S0163-5980), 2012, 45(3): 53-57.
[7] Leverich J, Kozyrakis C.On the energy (in) efficiency of hadoop clusters[J]. ACM SIGOPS Operating Systems Review (S0163-5980), 2010, 44(1): 61-65.
[8] Lang W, Patel J M.Energy management for mapreduce clusters[J]. Proceedings of the VLDB Endowment (S2150-8097), 2010, 3(1-2): 129-139.
[9] Zaharia M, Borthakur D, Sarma J S, et al.Job scheduling for multi-user mapreduce clusters[M]// Tech. Rep. UCB/EECS-2009-55, 2009. Berkeley, USA: EECS Department, University of California, 2009.
[10] Ibanez E, Brinkman G, Lew D.Sub-hour solar data for power system modeling from static spatial variability analysis[M]. USA: National Renewable Energy Laboratory, 2012.
[11] Liu Z, Chen Y, Bash C, et al.Renewable and cooling aware workload management for sustainable data centers[C]// ACM SIGMETRICS Performance Evaluation Review. USA: ACM, 2012, 40(1): 175-186.
[12] Li C, Zhang W, Cho C B, et al.SolarCore: Solar energy driven multi-core architecture power management[C]// High Performance Computer Architecture (HPCA), 2011 IEEE 17th International Symposium on. USA: IEEE, 2011: 205-216.
[13] Li C, Qouneh A, Li T. iSwitch: coordinating and optimizing renewable energy powered server clusters[C]// Computer Architecture (ISCA), 2012 39th Annual International Symposium on. USA: IEEE, 2012: 512-523.
[14] Sharma N, Barker S, Irwin D, et al.Blink: managing server clusters on intermittent power[C]// ACM SIGARCH Computer Architecture News. USA: ACM, 2011, 39(1): 185-198.

[1]	张森, 张孟炎, 邵敬平, 普杰信. 基于随机策略搜索的多机三维路径规划方法[J]. 系统仿真学报, 2022, 34(6): 1286-1295.
[2]	龙艳琴, 乔贵方, 宋光明, 张颖, 程琳琳. 二维变刚度蛇形机器人的运动仿真与性能分析[J]. 系统仿真学报, 2022, 34(4): 759-767.
[3]	白燕, 武璐璐, 贺引娥, 王玉英. 基于动态温度调控的空调系统能耗预测[J]. 系统仿真学报, 2022, 34(2): 366-375.
[4]	胡蓉, 董钰明, 钱斌. 基于探路者算法的绿色有限缓冲区流水线调度[J]. 系统仿真学报, 2021, 33(6): 1384-1396.
[5]	李文翔, 李晔, 董洁霜, 李一鸣. 引入碳交易机制的新能源汽车发展路径研究[J]. 系统仿真学报, 2021, 33(6): 1451-1465.
[6]	李霏, 苏中. 基于自适应免疫优化的污水处理过程控制[J]. 系统仿真学报, 2021, 33(12): 3012-3020.
[7]	张新艳, 周雨晴. 多载量小车节能调度的多目标动态规划算法[J]. 系统仿真学报, 2020, 32(4): 612-619.
[8]	孟庆龙, 张志博, 熊成. 变风量空调系统变水温策略运行效果仿真研究[J]. 系统仿真学报, 2019, 31(8): 1617-1626.
[9]	王华进, 万萌, 韩锐, 任玮, 张海明, 黎建辉. 构建自动演进的天文大数据负载模型[J]. 系统仿真学报, 2018, 30(9): 3293-3305.
[10]	王修岩, 刘艳敏, 张革文, 李宗帅, 林家泉. 基于PSO和CRO联合算法的飞机客舱能耗预测[J]. 系统仿真学报, 2018, 30(8): 3074-3081.
[11]	王修岩, 刘艳敏, 张革文, 李宗帅, 林家泉. 基于改进协同PSO的神经网络飞机客舱能耗预测[J]. 系统仿真学报, 2018, 30(4): 1535-1541.
[12]	李慧, 杨少刚, 刘凤英. GSHP机组数学建模及仿真应用[J]. 系统仿真学报, 2018, 30(3): 914-921.
[13]	胡可臻, 吴建平, 刘明宇. 实测与问卷驾驶行为数据视角的电动汽车能耗建模[J]. 系统仿真学报, 2018, 30(11): 4106-4114.
[14]	张晓星, 王艳, 严大虎, 纪志成. 改进的混合蛙跳算法求解柔性作业车间调度[J]. 系统仿真学报, 2017, 29(9): 2093-2099.
[15]	邝祝芳, 陈志刚. CRAHNs中干扰和能耗有效的联合功率控制路由算法仿真研究[J]. 系统仿真学报, 2017, 29(7): 1464-1471.