原位可视化研究与应用进展

摘要/Abstract

摘要： 科学可视化是分析数值模拟数据的最有效手段。随着高性能计算机运算能力的不断提升,数值模拟问题的规模迅速增长,但IO带宽、存储系统容量和后处理的可视化模式很难满足超大规模并行数值模拟数据的可视化分析需求。原位可视化在运行数值模拟任务的高性能计算机系统上进行可视化计算和绘制,克服IO和存储容量的限制,实现了在线数据可视化。从数据组织管理、并行可视化算法、交互、集成框架与软件、应用案例等方面介绍了原位可视化近年研究和应用进展,对原位可视化面临的技术挑战以及未来发展趋势进行了分析。

关键词: 原位可视化, 大规模并行处理, 软件框架, 可视化软件

Abstract: Scientific-Visualization is the most effective means of analyzing numerical simulation data. With the continuous improvement of the high performance computing technology, numerical simulation scale is growing rapidly. It is very difficult to analyze large scale parallel numerical simulation data efficiently due to the limits of IO bandwidth, storage system capacity and post processing pattern of visualization. In-situ visualization utilized the same computer system that ran the simulation tasks to operate visual computing and rendering, so it could overcome the limits of IO and storage system. The advances of in-situ visualization in recent years were reviewed, including data management, large-scale parallel visualization algorithm, interaction, application framework, visualization software and some use cases. The challenges and trends of in-situ visualization were discussed.

Key words: in-situ visualization, extreme-scale parallel processing, framework, visualization software

中图分类号:

TP391

王昉, 李思昆, 赵丹, 曾亮. 原位可视化研究与应用进展[J]. 系统仿真学报, 2015, 27(10): 2589-2599.

Wang Fang, Li Sikun, Zhao Dan, Zeng Liang. Advances in Research and Application of In-situ Visualization[J]. Journal of System Simulation, 2015, 27(10): 2589-2599.

参考文献

[1] Mccormick B H, Defanti T A, Brown M D.Visualization in scientific computing[J]. Acm Sigbio Newsletter (S0163-5697), 1988, 10(1): 15-21.
[2] Wong P C, Shen H-W, Johnson C R, et al. The Top 10 Challenges in Extreme-Scale Visual Analytics[J]. IEEE Computer Graphics and Applications (S0272-1716), 2012, 32(4): 63-67.
[3] Ma K-L, Wang C, Yu H, et al. In-situ processing and visualization for ultrascale simulations[J]. Journal of Physics: Conference Series (S1742-6588), 2007, 78: 012043
[4] 李思昆, 蔡勋, 王文珂, 等. 大规模流场科学计算可视化 [M]. 北京: 国防工业出版社, 2013.
[5] Ma K L.In Situ Visualization at Extreme Scale: Challenges and Opportunities[J]. IEEE Computer Graphics and Applications (S0272-1716), 2009, 29(6): 14-19.
[6] 单桂华, 田东, 谢茂金, 等. 千万亿次科学计算的原位可视化[J]. 计算机辅助设计与图形学学报, 2013, 25(3): 286-293.
[7] Rivi M, Calori L, Muscianisi G, et al. In-situ Visualization: State-of-the-art and Some Use Cases[J]. PRACE White Paper, 2012
[8] Tu T, Yu H, Ramirez-Guzman L, et al. From mesh generation to scientific visualization — an end-to-end approach to parallel supercomputing[C]// SC2006. Tampa, FL,USA: IEEE, 2006: 12.
[9] Duque E P, Hiepler D, Stone C P, et al. IFDT-Intelligent In-Situ Feature Detection, Extraction, Tracking and Visualization for Turbulent Flow Simulations[C]// ICCFD7. Hawaii,USA:ICCFD, 2012: 7.
[10] VisIt https://wci.llnl.gov/simulation/computer-codes/visit/
[11] Whitlock B, Favre J M, Meredith J S.Parallel In Situ Coupling of Simulation with a Fully Featured Visualization System[C]// Eurographics Symposium on Parallel Graphics and Visualization, 2011. Llandudno, UK: Eurographics Association, 2011: 101-109
[12] Paraview. http://www.paraview.org/
[13] Fabian N, Moreland K, Thompson D, et al. The ParaView coprocessing library: A scalable, general purpose in situ visualization library[C]// Large Data Analysis and Visualization, 2011 IEEE Symposium on. USA: IEEE, 2011: 89-96.
[14] Rogers D, Moreland K D, Oldfield R A, ,et al. Data co-processing for extreme scale analysis level II ASC milestone. Data co-processing for extreme scale analysis level II ASC milestone (4745) [R]. Albuquerque: Sandia National Laboratories, 2013
[15] Vtkhttp://www.kitware.com/opensource/vtk.html.
[16] Zheng F, Klasky S, Podhorszki N, et al. FlexIO: I/O Middleware for Location-Flexible Scientific Data Analytics[C]// Parallel and Distributed Processing, 2013 IEEE 27t h International Symposium on. USA: IEEE, 2013: 320-331.
[17] Lofstead J F, Klasky S, Schwan K, et al. Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS)[C]// Challenges of Large Applications in Distributed Environments, 2008. Boston,MA,USA: ACM, 2008: 15-24
[18] Docan C, Parashar M, Klasky S.DataSpaces: an interaction and coordination framework for coupled simulation workflows[J]. Cluster Computing (S1386-7857), 2012, 15(2): 163-181.
[19] Docan C, Parashar M, Cummings J, et al. Moving the Code to the Data - Dynamic Code Deployment Using ActiveSpaces[C]// Parallel & Distributed Processing Symposium (IPDPS), 2011 IEEE International. USA: IEEE, 2011: 758-769.
[20] Abbasi H, Wolf M, Eisenhauer G, et al. DataStager: scalable data staging services for petascale applications[J]. Cluster Computing(S1386-7857), 2010, 13(3): 277-290.
[21] Jerome Soumagne, John Biddiscombe, Clarke J.An HDF5 MPI Virtual File Driver for Parallel In-situ Post-processing[J]. Lecture Notes in Computer Science (S0302-9743), 2010, 6305: 62-71.
[22] Biddiscombe J, Soumagne J, Oger G, et al. Parallel Computational Steering and Analysis for HPC Applications using a ParaView Interface and the HDF5 DSM Virtual File Driver[C]. Eurographics Symposium on Parallel Graphics and Visualization, 2011. Llandudno: Eurographics Association, 2011: 91-100
[23] Soumagne J, Biddiscombe J, Esnard A.Data redistribution using one-sided transfers to in-memory HDF5 files[J]. Lecture Notes in Computer Science (S0302-9743), 2011, 6960: 198-207.
[24] Biddiscombe J, Soumagne J, Oger G, et al.Parallel Computational Steering for HPC Applications Using HDF5 Files in Distributed Shared Memory[J]. IEEE Transactions on Visualization and Computer Graphics (S1077-2626), 2012, 18(6): 852-864.
[25] Soumagne J.An In-situ Visualization Approach for Parallel Coupling and Steering of Simulations through Distributed Shared Memory Files [D]. France: Université Sciences et Technologies - Bordeaux I, 2012.
[26] Lakshminarasimhan S, Shah N, Ethier S, et al. ISABELA for effective in situ compression of scientific data[J]. Concurrency and Computation: Practice and Experience (S1532-0634), 2013, 25(4): 524-540.
[27] Schendel E R, Pendse S V, Jenkins J, et al. ISOBAR hybrid compression-I/O interleaving for large-scale parallel I/O optimization[C]// Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing. USA: ACM, 2012.
[28] Kogge P M.ExaScale Computing Study: Technology Challenges in Achieving Exascale Systems[M]. Darpa Information Processing Techniques Office, 2008.
[29] H Y, C W, Rw G, et al. In Situ Visualization for Large-Scale Combustion Simulations[J]. IEEE Computer Graphics and Applications (S0272-1716), 2010, 30(3): 45-57.
[30] Guo H, Zhang J, Liu R, et al. Advection-Based Sparse Data Management for Visualizing Unsteady Flow[J]. IEEE Transactions on Visualization and Computer Graphics (S1077-2626), 2014, (1): 2555-2564.
[31] Nouanesengsy B, Lee T-Y, Shen H-W.Load-balanced parallel streamline generation on large scale vector fields[J]. IEEE Transactions on Visualization and Computer Graphics,(S1077-2626) 2011, 17(12): 1785-1794.
[32] Chen C-M, Nouanesengsy B, Lee T-Y, et al. Flow-guided file layout for out-of-core pathline computation[C]// Large Data Analysis and Visualization, 2012 IEEE Symposium on. USA: IEEE, 2012: 109-112.
[33] Chen C-M, Xu L, Lee T, et al. A flow-guided file layout for out-of-core streamline computation[C]// Pacific Visualization Symposium, 2012 IEEE. USA: IEEE, 2012: 145-152.
[34] Camp D, Garth C, Childs H, et al. Streamline integration using MPI-hybrid parallelism on a large multicore architecture[J]. IEEE Transactions on Visualization and Computer Graphics (S1077-2626), 2011, 17(11): 1702-1713.
[35] Camp D.GPU Acceleration of Particle AdvectionWorkloads in a Parallel, Distributed Memory Setting[C]// Proceedings of the 13th Eurographics Symposium on Parallel Graphics and Visualization, 2013. Girona: Eurog raphics Association, 2013: 1-8.
[36] Hagan R, Cao Y.Multi-gpu load balancing for in-situ visualization[C]// The 2011 International Conference on Parallel and Distributed Processing Techniques and Applications, 2011. San Diego, USA: UCMSS, 2011.
[37] Bethel E.Towards Exascale: High Performance Visualization and Analytics-Project Status Report [R]. Portland: DOE 2012 Exascale Research Conference and Workshop, 2013.
[38] Kageyama A, Yamada T.An approach to exascale visualization: Interactive viewing of in-situ visualization[J]. Computer Physics Communications (S0010-4655), 2014, 185(1): 79-85.
[39] Ahrens J, Jourdain S, O'leary P, et al. An image-based approach to extreme scale in situ visualization and analysis[C]// High Performance Computing, Networking, Storage and Analysis, SC14: International Conference for. IEEE, 2014. USA: IEEE, 2014.
[40] Lorendeau B, Fournier Y, Ribes A.In-Situ visualization in fluid mechanics using Catalyst: A case study for Code Saturne[C]// 2013 IEEE Symposium on Large-Scale Data Analysis and Visualization. USA: IEEE Computer Society, 2013: 53-57.
[41] 孟创斌. 高性能计算数据的远程交互可视化技术研究与实现 [D]. 长沙:国防科技大学, 2014.
[42] Coulaud O, Richart N, Esnard A.A Steering Environment for Online Parallel Visualization of Legacy Parallel Simulations[C]// Distributed Simulation and Real Time Applications, IEEE/ACM International Symposium on. USA: IEEE Computer Society, 2006: 7-14.
[43] Fan Z, Docan C, Parashar M, et al. Enabling In-situ Execution of Coupled Scientific Workflow on Multi-core Platform[C]// Parallel and Distributed Processing Symposium, International. USA: IEEE, 2012: 1352-1363.
[44] Dorier M, Sisneros R, Peterka T, et al. Damaris/Viz: A nonintrusive, adaptable and user-friendly in situ visualization framework[C]// Large-Scale Data Analysis and Visualization, 2013 IEEE Symposium on. USA: IEEE, 2013: 67-75.
[45] 路石, 孟创斌, 李思昆, 等. 高性能并行可视化服务器的资源管理技术研究[J]. 软件工程与应用, 2014, 3(5): 131-143.
[46] Fabian N, Moreland K, Thompson D, et al. The ParaView Coprocessing Library: A scalable, general purpose in situ visualization library[C]// Large Data Analysis and Visualization, 2011 IEEE Symposium on. USA: IEEE, 2011: 89-96.