Research on Technology of Data Storage and Access in High-throughput Simulation

doi:10.16182/j.issn1004731x.joss.201709019

Abstract

Abstract: Aiming at the massive data processing requirements of simulation application in the big data environment, the concept and reference structure of high-throughput simulation were proposed and defined in accordance with the architecture and technical characteristics of high-throughput big data computing. For the problem of data access bottleneck in high-throughput simulation, a high-throughput simulation data storage and access system was designed based on distributed memory file system, and the non-volatile memory was integrated to improve the throughput of data access. The experimental results of typical simulation applications show that the high-throughput simulation storage system with distributed memory file system and non-volatile memory can effectively improve the data storage and access performance of high-throughput simulation.

Key words: big data, high-throughput simulation, non-volatile memory, distributed file system

CLC Number:

TP391.1

Wang Zishuo, Zhai Yanlong, Tao Wenjun, Yang Hao, Zhang Han, Qing Duzheng. Research on Technology of Data Storage and Access in High-throughput Simulation[J]. Journal of System Simulation, 2017, 29(9): 2016-2024.

References

[1] 候彦庆. 计算机仿真技术的应用与发展趋势[J]. 信息通信, 2016 (2): 181-182.
(Hou Yanqing.The Application and Development Trend of Computer Simulation Technology[J]. Information & Communications, 2016 (2): 181-182.)
[2] 李宏亮. 基于Agent的复杂系统分布仿真 [D]. 长沙:国防科学技术大学, 2001.
(Li Hongliang.The Agent-Based Distributed Simulation for Complex Systems [D]. Changsha, China: National University of Defense Technology, 2001.)
[3] 李琼. 面向高性能计算的可扩展I/O体系结构研究与实现 [D]. 长沙: 国防科学技术大学, 2009.
(Li Qiong.Research on I/O Architecture and Implementing Technology for High Performance Computing [D]. Changsha, China: National University of Defense Technology, 2009.)
[4] May J M.Parallel I/O for high performance computing[M]. USA: Morgan Kaufmann Publisher, 2001: 178-179.
[5] 徐越. 面向高性能计算的分布式内存文件缓存系统 [D]. 武汉: 华中科技大学, 2013.
(Xu Yue.Distributed Memory File Caching System for High Performance Computing [D]. Wuhan, China: Huazhong University of Science and Technology, 2013.)
[6] 王士斌. 大数据应用技术浅析[J]. 计算机光盘软件与应用, 2014 (8): 96-97.
(Wang Shibin.Analysis of Application Technology of Large Data[J]. Computer CD Software and Applications, 2014 (8): 96-97.)
[7] Schroeder B, Gibson G A.A large-scale study of failures in high-performance computing systems[C]// International Conference on Dependable Systems and Networks. USA: IEEE, 2006: 249-258.
[8] 刘颖, 陈煜, 林林, 等. 高性能计算集群中的网络技术研究与实践[J]. 中国水利水电科学研究院学报, 2016, 14(2): 90-95.
(Liu Ying, Chen Yu, Lin Lin, et al.Research and practice of high performance computing cluster network technology[J]. Journal of China Institute of Water Resources and Hydropower Research, 2016, 14(2): 90-95.)
[9] Iwata K, Onosato M, Teramoto K, et al.A Modelling and Simulation Architecture for Virtual Manufacturing Systems[J]. CIRP Annals - Manufacturing Technology (S0007-8506), 1995, 44(1): 399-402.
[10] Liang S, Noronha R, Panda D K.Swapping to remote memory over infiniband: an approach using a high performance network block device[C]// Proc of the IEEE International Conference on Cluster Computing (Cluster2005). USA: IEEE, 2005.
[11] Grošelj B, Tropper C.The distributed simulation of clustered processes[J]. Distributed Computing (S0178-2770), 1991, 4(3): 111-121.
[12] 蔡涛, 牛德姣, 刘扬宽, 等. NVMMDS—一种面向非易失存储器的元数据管理方法[J]. 计算机研究与发展, 2013, 50(1): 69-79.
(Cai Tao, Niu Dejiao, Liu Yangkuan, et al.NVMMDS—Metadata Management Method Based on Non-Volatile Memory[J]. Journal of Computer Research and Development, 2013, 50(1): 69-79.)
[13] Manoj Sinha, Steven Hsu, Atila Alvandpour.High-Performance and Low-Voltage Sense-Amplifier Techniques for sub-90nm SRAM[C]// Proceedings of IEEE International SOC Conference. USA: IEEE, 2003.
[14] 陈怡然, 赵巍胜, 孙振宇, 等. 新型非易失性存储器[J]. 现代物理知识, 2015 (1): 41-46.
(Chen Yiran, Zhao Weisheng, Sun Zhenyu, et al.New Non-volatile Memory[J]. Modern Physics, 2015 (1): 41-46.)
[15] 何炎祥, 沈凡凡, 张军, 等. 新型非易失性存储器架构的缓存优化方法综述[J]. 计算机研究与发展, 2015, 52(6): 1225-1241.
(He Yanxiang, Shen Fanfan, Zhang Jun, et al.Cache Optimization Approaches of Emerging Non-Volatile Memory Architecture:A Survey[J]. Journal of Computer Research and Development, 2015, 52(6): 1225-1241.)