Research of Parallel Process Method of Ground Penetrating Radar (GPR) Data Based on Hadoop

doi:10.16182/j.issn1004731x.joss.201701017

Abstract

Abstract: The change of research way brought by big data and cloud computing has an impact on the research of GPR data processing. Filtering and analysis research for large scale GPR data was conducted by cloud computing platforms. Data preprocessing to GPR data was realized through flow computing framework Storm so as to solve the mismatch of GPR data format and input data format of Hadoop and realize parallel convolution and gain process based on Hadoop. The filtering result and performance index was analyzed. Experimental results show that the proposed method is accurate and effective on GPR data process, and it has significantly improved operating speed and parallel speed-up in comparison with previous method.

Key words: Hadoop, GPR data, Storm, convolution, gain process, parallel

CLC Number:

TP391

Liang Yincheng, Yuan Yuan, Yang Feng . Research of Parallel Process Method of Ground Penetrating Radar (GPR) Data Based on Hadoop[J]. Journal of System Simulation, 2017, 29(1): 120-128.

References

[1] 杨峰, 彭苏萍. 地质雷达探测原理与方法研究 [M]. 北京: 科学出版社, 2010.
(Yang Feng, Peng Suping.Research on principle and method of geological radar detection [M]. Beijing, China: Science Press, 2010.)
[2] Xiang X, Gao Y, Shang L, et al Parallel Text Categorization of Massive Text Based on Hadoop[J]. Computer Science(S1002-137X), 2011, 38(10): 184-188.
[3] Wang Fei, Vuk E, David B, et al.Large-scale multimodel mining for healthcare with MapReduce[C]// Proceedings of the 1st ACM International Health Informatics Symposium, USA: ACM, 2010.
[4] Malewicz Grzegorz, Austern Matthew H, Bik Aart J C, et al. Pregel: A system for large-scale graph processing[C]// Proceedings of the SIGMOD, USA: ACM, 2010: 135-146.
[5] 李贵兵, 金炜东, 蒋鹏, 等. 面向大规模监测数据的高铁故障诊断技术研究[J]. 系统仿真学报, 2014, 26(10): 2458-2464.
(Li Guibing, Jin Weidong, Jiang Peng, et al.Research on Fault Diagnosis Technology of High-speed Rail for Large Scale Monitoring Data[J]. Journal of System Simulation, 2014, 26(10): 2458-2464.)
[6] Gu T, Zuo C, Liao Q, et al.Improving MapReduce Performance by Data Prefetching in Heterogeneous or Shared Environments[J]. International Journal of Grid and Distributed Computing (S2005-4262), 2013, 6(5): 71-82.
[7] Shvachko K, Kuang H, Radia S, et al.The hadoop distributed file system[C]//Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on. USA: IEEE, 2010: 1-10.
[8] 周敏奇. Hadoop权威指南 [M]. 北京: 清华大学出版社, 2011: 198-215.
(Zhou Minqi.Hadoop the definitive guide [M]. Beijing: Tsinghua University Press, 2011: 198-215.)
[9] Jonathan Leibiusky, Gabriel Eisbruch, Dario Simonassi.Getting started with storm [M]. Sebastopol: O'reilly Press, 2012.
[10] Massie M L, Chun B N, Culler D E.The ganglia distributed monitoring system: design, implementation, and experience[J]. Parallel Computing (S0167-8191), 2004, 30(7):817-840.

[1]	Yin Wang, Feixiang Wang, Qianlai Sun. Vehicle Detection Method Based on Multi Scale Feature Fusion [J]. Journal of System Simulation, 2022, 34(6): 1219-1229.
[2]	Lifeng Zhang, Huiren Wang. Image Reconstruction of Electrical Capacitance Tomography Based on Convolutional Neural Network and Finite Element Simulation [J]. Journal of System Simulation, 2022, 34(4): 712-718.
[3]	Jiawen Yan, Jijie Huang, Lie Zhou, Changjin Chen, Qiang Wu, Jintao Zhao. Transmission Line Operation and Inspection Training Simulation Based on Multiple Time Scales and VR [J]. Journal of System Simulation, 2022, 34(2): 212-220.
[4]	Bangqian Ao, Sha Yang, Jinqing Linghu, zhenhuan Ye. Design of Fatigue Driving Detection System Based on Cascaded Neural Network [J]. Journal of System Simulation, 2022, 34(2): 323-333.
[5]	Lin Zhongwei, Li Xiuhe, Yao Yiping, Shen Yang, Dai Shaoqi, Shi Qianqian. Research on Simulation Object Interfaces Oriented to Plug-in-play Pattern [J]. Journal of System Simulation, 2021, 33(9): 2202-2214.
[6]	Huang Xiaodong, Xie Kongshu. Research and Implementation of a High Performance Distributed Object-oriented Simulation Engine [J]. Journal of System Simulation, 2021, 33(9): 2215-2226.
[7]	Gao Shengdong, Yu Xin. Co-simulation of Parallel Computing of Disc Temperature Distributed Parameter System Cooling Rate Control [J]. Journal of System Simulation, 2021, 33(8): 1839-1845.
[8]	Liu Yu, Xie Qiang. Technical Characteristics of Digital Twins and Application Prospects in the Field of Flight Testing [J]. Journal of System Simulation, 2021, 33(6): 1364-1373.
[9]	Zhao Ying, Lu Yao, Zhang Jian, Liang Qidi, Long Wei. Multi-view Human Action Recognition Based on Deep Neural Network [J]. Journal of System Simulation, 2021, 33(5): 1019-1030.
[10]	Tang Wenjie, Cheng Junwei, Yao Yiping, Zhu Feng. A Shared Memory Based Parallel Hierarchical Interest Matching Algorithm [J]. Journal of System Simulation, 2021, 33(5): 1086-1094.
[11]	Tong Weiguo, Pang Xuechun, Zhu Genghong. Gas-liquid Two-phase Flow Pattern Recognition Method Based on Convolutional Neural Network [J]. Journal of System Simulation, 2021, 33(4): 883-891.
[12]	Wang Buwei, Wang Min, Fan Qian, Wang Ya'nan, zhang hanwen, Yue Yunliang. Study on Prediction of Crystal Properties Based on Deep Learning [J]. Journal of System Simulation, 2021, 33(12): 2854-2863.
[13]	Ma Rong, Chen Qiurui, Zhang Han, Mei Zheng, Wang Rui, Wei Wei. Low Power Visual Odometry Technology Based on Monocular Depth Estimation [J]. Journal of System Simulation, 2021, 33(12): 3001-3011.
[14]	Huo Zhihao, Jin Weidong, Tang Peng. Single-frame Image Motion Parallax Key Point Estimation Combined with Self-supervised Learning [J]. Journal of System Simulation, 2021, 33(11): 2753-2759.
[15]	Wang Jianlin, Chen Sufang, Tie Ming, Liu Jing, Zhang Jun. High Performance Simulation Method and Test Platform for Full Envelop of Flight Vehicle [J]. Journal of System Simulation, 2021, 33(10): 2298-2306.