Data Simulation Testing Framework for Complex Process Equipment Software

doi:10.16182/j.issn1004731x.joss.22-0854

Abstract

Abstract:

Due to the complex task, tight coupling, strict timing, and a large amount of interchange data, the technical threshold of automated testing of bus communication equipment software is high, and the implementation is difficult. The ideas of data-driven testing and keyword-driven testing are introduced, and a data simulation testing framework is proposed. Configuration rules are formulated and implemented in the framework. Testers can simulate peripheral data for complex process equipment software and implement automated testing by only focusing on the task analysis, and configuring interchange data and keywords. There is no need to develop test scripts, which reduces the technical threshold of automated testing. Application results show that the framework can perform more task scenarios than manual and physical testing. It also can find the deep software defects related to the task processes.

Key words: data simulation, automated testing, testing framework, business process, equipment software, data-driven, keyword-driven

CLC Number:

TP391.9

Zhang Jinkun, Shi Longfei, Hu Chi, Zhang Hao, Yang Yonghui. Data Simulation Testing Framework for Complex Process Equipment Software[J]. Journal of System Simulation, 2023, 35(12): 2512-2526.

Figures/Tables 15

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Table 1

Fig. 11

Table 2

Fig. 12

Table 3

References 34

1	Zhang Jinkun, Yang Hongqing, Sun Zhe, et al. Automated Testing of Multi-UDP Interface Equipment Software Based on Scenario and Data Driven[C]//2021 IEEE 4th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC). Piscataway, NJ, USA: IEEE, 2021: 867-875.
2	Zhan Guang, Gong Zheng, Quanhui Lü, et al. Flight Test of Autonomous Formation Management for Multiple Fixed-wing UAVs Based on Missile Parallel Method[J]. Drones, 2022, 6(5): 99.
3	Li Wenxing, Fang Yu, He Xiaojun, et al. Analysis and Design of Testing Supporting System of Flight Track Processing Software[C]//International Conference on Intelligent Systems, Communications, and Computer Networks (ISCCN 2022). Bellingham, WA, USA: SPIE, 2022: 123320O.
4	吴枫, 刘秀罗, 王佳, 等. 航天发射可视化仿真分析技术与应用研究[J]. 系统仿真学报, 2022, 34(4): 856-869.
	Wu Feng, Liu Xiuluo, Wang Jia, et al. Research on Space Launch Visualization Simulation Analysis Technology and Application[J]. Journal of System Simulation, 2022, 34(4): 856-869.
5	冯洋, 夏志龙, 郭安, 等. 自动驾驶软件测试技术研究综述[J]. 中国图象图形学报, 2021, 26(1): 13-27.
	Feng Yang, Xia Zhilong, Guo An, et al. Survey of Testing Techniques of Autonomous Driving Software[J]. Journal of Image and Graphics, 2021, 26(1): 13-27.
6	孙剑, 黄润涵, 李霖, 等. 智能汽车环境感知与规划决策一体化仿真测试平台[J]. 系统仿真学报, 2020, 32(2): 236-246.
	Sun Jian, Huang Runhan, Li Lin, et al. Integrated Simulation Test Platform for Environment Perception and Planning Decision of Intelligent Vehicle[J]. Journal of System Simulation, 2020, 32(2): 236-246.
7	Ingalls R G. Introduction to Simulation[C]//Proceedings of the 2011 Winter Simulation Conference (WSC). Piscataway, NJ, USA: IEEE, 2011: 1374-1388.
8	Garousi V, Felderer M, Çağrı Murat Karapıçak, et al. Testing Embedded Software: A Survey of the Literature[J]. Information and Software Technology, 2018, 104: 14-45.
9	Park G, Wil M P van der Aalst. Towards Reliable Business Process Simulation: A Framework to Integrate ERP Systems[C]//Enterprise, Business-Process and Information Systems Modeling. Cham: Springer International Publishing, 2021: 112-127.
10	Garousi V, Elberzhager F. Test Automation: Not Just for Test Execution[J]. IEEE Software, 2017, 34(2): 90-96.
11	程海南. 视讯系统的软件自动化测试应用与研究[D]. 南京: 东南大学, 2015.
	Cheng Hainan. Application and Researchment of Software Autotest Used in Video Conference System[D]. Nanjing: Southeast University, 2015.
12	章鸽鸽. 基于关键字驱动的自动化测试工具的设计与实现[D]. 合肥: 安徽大学, 2018.
	Zhang Gege. Design and Implementation of A Keyword-driven Automation Test Tool[D]. Hefei: Anhui University, 2018.
13	Laukkanen P. Data-driven and Keyword-driven Test Automation Frameworks[D]. Espoo: Helsinki University of Technology, 2006.
14	Mirza A M, Khan M N A, Wagan R A, et al. ContextDrive: Towards a Functional Scenario-based Testing Framework for Context-aware Applications[J]. IEEE Access, 2021, 9: 80478-80490.
15	Yang Chenwei, Vyatkin V. On Automated Co-simulation Testing of Functional Requirements for Distributed Substation Automation Systems[C]//IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society. Piscataway, NJ, USA: IEEE, 2018: 3576-3581.
16	Amir K C, Goulart A, Kantola R. Keyword-driven Security Test Automation of Customer Edge Switching (CES) Architecture[C]//2016 8th International Workshop on Resilient Networks Design and Modeling (RNDM). Piscataway, NJ, USA: IEEE, 2016: 216-223.
17	Shostack A. Threat Modeling: Designing for Security[M]. Indianapolis, IN: John Wiley and Sons, 2014.
18	Hu Ting, Li Qin, Huang Xin, et al. Design and Implementation of an Automated Test Framework for Electric Power Data Network Based on the Unified Resource Scheduling[C]//2019 IEEE 8th International Conference on Advanced Power System Automation and Protection (APAP). Piscataway, NJ, USA: IEEE, 2019: 1553-1557.
19	Yamini A, Rajesh M. Automation Framework for Verification of LAMPS and DCS in Airbag Control Unit (ACU)[C]//2017 International Conference on Inventive Communication and Computational Technologies (ICICCT). Piscataway, NJ, USA: IEEE, 2017: 385-389.
20	Cobar M J, Montalvo C J. The Facility for Aerospace Systems and Technology Simulation: FASTSim-an Open Source Configurable Software in the Loop Simulation Environment[C]//AIAA AVIATION 2022 Forum. Reston, VA, USA: AIAA, 2022: AIAA 2022-3943.
21	Reddy B S, Padmavathi, Maruthamma P, et al. Automation of Testing Military Avionics Mission Critical Software[C]//2018 Second International Conference on Electronics, Communication and Aerospace Technology (ICECA). Piscataway, NJ, USA: IEEE, 2018: 1378-1383.
22	Hametner R, Winkler D, Zoitl A. Agile Testing Concepts Based on Keyword-driven Testing for Industrial Automation Systems[C]//IECON 2012-38th Annual Conference on IEEE Industrial Electronics Society. Piscataway, NJ, USA: IEEE, 2012: 3727-3732.
23	Yang Shuaishuai, Yu Zhengwei, Liu Bin, et al. An Automatic Testing Framework for Embedded Software[C]//2017 12th International Conference on Computer Science and Education (ICCSE). Piscataway, NJ, USA: IEEE, 2017: 269-274.
24	王子豪. 基于需求模型的航天软件测试自动化方法研究[D]. 北京: 中国运载火箭技术研究院, 2019.
	Wang Zihao. Research on Test Automation Method of Aerospace Software Based on Requirement Model[D]. Beijing: China Academy of Launch Vehicle Technology, 2019.
25	Khaliq Z, Farooq S U, Khan D A. A Deep Learning-based Automated Framework for Functional User Interface Testing[J]. Information and Software Technology, 2022, 150: 106969.
26	Shahamiri S R, Kadir W M N W, Ibrahim S, et al. An Automated Framework for Software Test Oracle[J]. Information and Software Technology, 2011, 53(7): 774-788.
27	Zhang Jinkun, Yang Hongqing, Sun Zhe, et al. Automatic Generation of Interface Test Data Based on Protocol Modeling and Constraint Coverage[C]//2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). Piscataway, NJ, USA: IEEE, 2021: 2322-2327.
28	Arezoo Yazdani Seqerloo, Mohammad Javad Amiri, Parsa Saeed, et al. Automatic Test Cases Generation From Business Process Models[J]. Requirements Engineering, 2019, 24(1): 119-132.
29	Paiva A C R, Flores N H, Faria João P, et al. End-to-end Automatic Business Process Validation[J]. Procedia Computer Science, 2018, 130: 999-1004.
30	丁明, 张书玲, 张琛. 业务流程建模与测试方法研究[J]. 西安交通大学学报, 2016, 50(3): 127-133.
	Ding Ming, Zhang Shuling, Zhang Chen. Modeling and Testing Methods of Business Process[J]. Journal of Xi'an Jiaotong University, 2016, 50(3): 127-133.
31	Böhmer Kristof, Rinderle-Ma S. Automatic Business Process Test Case Selection: Coverage Metrics, Algorithms, and Performance Optimizations[J]. International Journal of Cooperative Information Systems, 2016, 25(4): 1740002.
32	Oditis I, Bicevskis J. Asynchronous Runtime Verification of Business Processes[C]//2015 7th International Conference on Computational Intelligence, Communication Systems and Networks. Piscataway, NJ, USA: IEEE, 2015: 103-108.
33	International Organization for Standardization. Software and Systems Engineering-software Testing-part 5: Keyword-driven Testing: [S]. Vernier, Geneva, Switzerland: International Organization for Standardization, 2016.
34	He Zhonghai, Zhang Xiang, Zhu Xiangyin. Design and Implementation of Automation Testing Framework Based on Keyword Driven[J]. Applied Mechanics and Materials, 2014, 602-605: 2142-2146.

业务模块	交换数据组数	交换数据
业务模块	交换数据组数	指令数	激励数
BM1	0	0	0
BM2	1	1	2
BM3	n	n	n
BM4	2	2	3
BM5	0	0	0
BM6	1	1	1
BM7	1	1	2
BM8	0	0	0
BM9	1	1	2
BM10	1	1	1
BM11	0	0	0

No.	流程入口	BM1	BM2	BM3	TC1	BM4	BM5	BM6	BM7	BM8	TC2	TC3	BM9	BM10	BM11	流程出口
1	单击“开始”按钮	S	F	―	―	―	―	―	―	―	―	―	―	―	―	终止
2		S	S	F	―	―	―	―	―	―	―	―	―	―	―	终止
3		S	S	F	否	―	―	―	―	―	―	―	―	―	―	终止
4		S	S	S	是	F	―	―	―	―	―	―	―	―	―	终止
5		S	S	S	是	S	S	F	F	―	―	―	―	―	―	终止
6		S	S	S	是	S	S	F	S	S	是	―	S	S	S	结束
7		S	S	S	是	S	S	F	S	S	否	是	F	―	―	终止
8		S	S	S	是	S	S	F	S	S	否	是	S	F	S	结束
9		S	S	S	是	S	S	F	S	S	否	是	S	S	S	结束
10		S	S	S	是	S	S	S	F	―	―	―	―	―	―	终止
11		S	S	S	是	S	S	S	S	S	是	―	S	S	S	结束
12		S	S	S	是	S	S	S	S	S	否	是	F	―	―	终止
13		S	S	S	是	S	S	S	S	S	否	是	S	F	S	结束
14		S	S	S	是	S	S	S	S	S	否	是	S	S	S	结束

No.	问题类别	问题级别	问题描述
1	设计缺陷	严重	设计时没有考虑信号触发、时间等能够结束流程的约束
2	设计缺陷	严重	没有设计时序参数和流程之间的约束
3	代码缺陷	严重	处理多包指令时，软件只判断了第一包数据的有效性
4	设计缺陷	严重	处理数字信号时，软件未过滤毛刺信号
5	代码缺陷	严重	第n包参数下载失败时，软件只重传一次参数下载指令
6	设计缺陷	一般	当信号或者动作异常时，主控软件没有设计流程轨迹灯的相应显示状态

[1]	Qiping Lai, Tannan Xiao, Dongsheng Li, Chen Shen. Low Voltage Ride-through Modeling for Wind Turbines Based on Neural ODEs [J]. Journal of System Simulation, 2022, 34(12): 2546-2556.
[2]	Zeng Bi, Fang Xiao, Kong Deshuai, Song Xiangxiang, Jia Zhengxuan, Lin Tingyu. A Data-Driven Modeling Method for Game Adversity Agent [J]. Journal of System Simulation, 2021, 33(12): 2838-2845.
[3]	Wang Yuanyuan, Duan Yuxin, Song Guangzhao. Research on Configurable Simulation Integration Technology for Equipment Software [J]. Journal of System Simulation, 2020, 32(11): 2093-2099.
[4]	Wang Jie, Chen Bin, Yuan Peng, Ma Liang, Qiu Xiaogang. Study on Data-driven ORCA Preference Velocity [J]. Journal of System Simulation, 2019, 31(12): 2731-2739.
[5]	Ma Tianyu, Wang Yalin, Shen Kun, Liu Jinping. Prediction Model of Particle Size Distribution in Bauxite Continuous Ball Milling Process [J]. Journal of System Simulation, 2018, 30(2): 414-421.
[6]	Lu Yingbo, Qian Xiaochao, Chen Wei, He Shu, Lu Zhifeng. Research on Construction Method of Data-Driven Equipment Effectiveness Evaluation Model [J]. Journal of System Simulation, 2018, 30(12): 4587-4595.
[7]	Huang Ying, Li Kangshun, Li Wei, He Keqing. Business Process Modeling Algorithm Based on Ontology Language [J]. Journal of System Simulation, 2017, 29(10): 2282-2290.
[8]	Chen Guodong, Wang Jiexiong, Chen Yi. Study on Personalized Data Driven Method of Surface of 3D Liver Models [J]. Journal of System Simulation, 2015, 27(2): 270-278.