复杂流程装备软件数据仿真测试框架

doi:10.16182/j.issn1004731x.joss.22-0854

摘要/Abstract

摘要：

针对基于总线通信的装备软件由于业务复杂、耦合紧密、时序严格、数据交换量大导致的自动化测试技术门槛高、实施难度大的问题，引入数据驱动测试和关键字驱动测试的思想，提出了一种数据仿真测试框架，制定了配置规则并在框架中实现。测试人员只需专注于业务分析，配置交换数据和关键字，就可以对复杂流程装备软件进行外围数据仿真，实施自动化测试，无需开发测试脚本，降低了自动化测试的技术门槛。结果表明：与手动测试和实物测试相比，该框架可以执行更多的业务场景，能发现业务流程相关的深层次的软件缺陷。

关键词: 数据仿真, 自动化测试, 测试框架, 业务流程, 装备软件, 数据驱动, 关键字驱动

Abstract:

Due to the complex task, tight coupling, strict timing, and a large amount of interchange data, the technical threshold of automated testing of bus communication equipment software is high, and the implementation is difficult. The ideas of data-driven testing and keyword-driven testing are introduced, and a data simulation testing framework is proposed. Configuration rules are formulated and implemented in the framework. Testers can simulate peripheral data for complex process equipment software and implement automated testing by only focusing on the task analysis, and configuring interchange data and keywords. There is no need to develop test scripts, which reduces the technical threshold of automated testing. Application results show that the framework can perform more task scenarios than manual and physical testing. It also can find the deep software defects related to the task processes.

Key words: data simulation, automated testing, testing framework, business process, equipment software, data-driven, keyword-driven

中图分类号:

TP391.9

张进坤,史龙飞,胡驰等 . 复杂流程装备软件数据仿真测试框架[J]. 系统仿真学报, 2023, 35(12): 2512-2526.

Zhang Jinkun,Shi Longfei,Hu Chi,et al . Data Simulation Testing Framework for Complex Process Equipment Software[J]. Journal of System Simulation, 2023, 35(12): 2512-2526.

图/表 15

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

表1

图11

表2

图12

表3

参考文献 34

1	Zhang Jinkun, Yang Hongqing, Sun Zhe, et al. Automated Testing of Multi-UDP Interface Equipment Software Based on Scenario and Data Driven[C]//2021 IEEE 4th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC). Piscataway, NJ, USA: IEEE, 2021: 867-875.
2	Zhan Guang, Gong Zheng, Quanhui Lü, et al. Flight Test of Autonomous Formation Management for Multiple Fixed-wing UAVs Based on Missile Parallel Method[J]. Drones, 2022, 6(5): 99.
3	Li Wenxing, Fang Yu, He Xiaojun, et al. Analysis and Design of Testing Supporting System of Flight Track Processing Software[C]//International Conference on Intelligent Systems, Communications, and Computer Networks (ISCCN 2022). Bellingham, WA, USA: SPIE, 2022: 123320O.
4	吴枫, 刘秀罗, 王佳, 等. 航天发射可视化仿真分析技术与应用研究[J]. 系统仿真学报, 2022, 34(4): 856-869.
	Wu Feng, Liu Xiuluo, Wang Jia, et al. Research on Space Launch Visualization Simulation Analysis Technology and Application[J]. Journal of System Simulation, 2022, 34(4): 856-869.
5	冯洋, 夏志龙, 郭安, 等. 自动驾驶软件测试技术研究综述[J]. 中国图象图形学报, 2021, 26(1): 13-27.
	Feng Yang, Xia Zhilong, Guo An, et al. Survey of Testing Techniques of Autonomous Driving Software[J]. Journal of Image and Graphics, 2021, 26(1): 13-27.
6	孙剑, 黄润涵, 李霖, 等. 智能汽车环境感知与规划决策一体化仿真测试平台[J]. 系统仿真学报, 2020, 32(2): 236-246.
	Sun Jian, Huang Runhan, Li Lin, et al. Integrated Simulation Test Platform for Environment Perception and Planning Decision of Intelligent Vehicle[J]. Journal of System Simulation, 2020, 32(2): 236-246.
7	Ingalls R G. Introduction to Simulation[C]//Proceedings of the 2011 Winter Simulation Conference (WSC). Piscataway, NJ, USA: IEEE, 2011: 1374-1388.
8	Garousi V, Felderer M, Çağrı Murat Karapıçak, et al. Testing Embedded Software: A Survey of the Literature[J]. Information and Software Technology, 2018, 104: 14-45.
9	Park G, Wil M P van der Aalst. Towards Reliable Business Process Simulation: A Framework to Integrate ERP Systems[C]//Enterprise, Business-Process and Information Systems Modeling. Cham: Springer International Publishing, 2021: 112-127.
10	Garousi V, Elberzhager F. Test Automation: Not Just for Test Execution[J]. IEEE Software, 2017, 34(2): 90-96.
11	程海南. 视讯系统的软件自动化测试应用与研究[D]. 南京: 东南大学, 2015.
	Cheng Hainan. Application and Researchment of Software Autotest Used in Video Conference System[D]. Nanjing: Southeast University, 2015.
12	章鸽鸽. 基于关键字驱动的自动化测试工具的设计与实现[D]. 合肥: 安徽大学, 2018.
	Zhang Gege. Design and Implementation of A Keyword-driven Automation Test Tool[D]. Hefei: Anhui University, 2018.
13	Laukkanen P. Data-driven and Keyword-driven Test Automation Frameworks[D]. Espoo: Helsinki University of Technology, 2006.
14	Mirza A M, Khan M N A, Wagan R A, et al. ContextDrive: Towards a Functional Scenario-based Testing Framework for Context-aware Applications[J]. IEEE Access, 2021, 9: 80478-80490.
15	Yang Chenwei, Vyatkin V. On Automated Co-simulation Testing of Functional Requirements for Distributed Substation Automation Systems[C]//IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society. Piscataway, NJ, USA: IEEE, 2018: 3576-3581.
16	Amir K C, Goulart A, Kantola R. Keyword-driven Security Test Automation of Customer Edge Switching (CES) Architecture[C]//2016 8th International Workshop on Resilient Networks Design and Modeling (RNDM). Piscataway, NJ, USA: IEEE, 2016: 216-223.
17	Shostack A. Threat Modeling: Designing for Security[M]. Indianapolis, IN: John Wiley and Sons, 2014.
18	Hu Ting, Li Qin, Huang Xin, et al. Design and Implementation of an Automated Test Framework for Electric Power Data Network Based on the Unified Resource Scheduling[C]//2019 IEEE 8th International Conference on Advanced Power System Automation and Protection (APAP). Piscataway, NJ, USA: IEEE, 2019: 1553-1557.
19	Yamini A, Rajesh M. Automation Framework for Verification of LAMPS and DCS in Airbag Control Unit (ACU)[C]//2017 International Conference on Inventive Communication and Computational Technologies (ICICCT). Piscataway, NJ, USA: IEEE, 2017: 385-389.
20	Cobar M J, Montalvo C J. The Facility for Aerospace Systems and Technology Simulation: FASTSim-an Open Source Configurable Software in the Loop Simulation Environment[C]//AIAA AVIATION 2022 Forum. Reston, VA, USA: AIAA, 2022: AIAA 2022-3943.
21	Reddy B S, Padmavathi, Maruthamma P, et al. Automation of Testing Military Avionics Mission Critical Software[C]//2018 Second International Conference on Electronics, Communication and Aerospace Technology (ICECA). Piscataway, NJ, USA: IEEE, 2018: 1378-1383.
22	Hametner R, Winkler D, Zoitl A. Agile Testing Concepts Based on Keyword-driven Testing for Industrial Automation Systems[C]//IECON 2012-38th Annual Conference on IEEE Industrial Electronics Society. Piscataway, NJ, USA: IEEE, 2012: 3727-3732.
23	Yang Shuaishuai, Yu Zhengwei, Liu Bin, et al. An Automatic Testing Framework for Embedded Software[C]//2017 12th International Conference on Computer Science and Education (ICCSE). Piscataway, NJ, USA: IEEE, 2017: 269-274.
24	王子豪. 基于需求模型的航天软件测试自动化方法研究[D]. 北京: 中国运载火箭技术研究院, 2019.
	Wang Zihao. Research on Test Automation Method of Aerospace Software Based on Requirement Model[D]. Beijing: China Academy of Launch Vehicle Technology, 2019.
25	Khaliq Z, Farooq S U, Khan D A. A Deep Learning-based Automated Framework for Functional User Interface Testing[J]. Information and Software Technology, 2022, 150: 106969.
26	Shahamiri S R, Kadir W M N W, Ibrahim S, et al. An Automated Framework for Software Test Oracle[J]. Information and Software Technology, 2011, 53(7): 774-788.
27	Zhang Jinkun, Yang Hongqing, Sun Zhe, et al. Automatic Generation of Interface Test Data Based on Protocol Modeling and Constraint Coverage[C]//2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). Piscataway, NJ, USA: IEEE, 2021: 2322-2327.
28	Arezoo Yazdani Seqerloo, Mohammad Javad Amiri, Parsa Saeed, et al. Automatic Test Cases Generation From Business Process Models[J]. Requirements Engineering, 2019, 24(1): 119-132.
29	Paiva A C R, Flores N H, Faria João P, et al. End-to-end Automatic Business Process Validation[J]. Procedia Computer Science, 2018, 130: 999-1004.
30	丁明, 张书玲, 张琛. 业务流程建模与测试方法研究[J]. 西安交通大学学报, 2016, 50(3): 127-133.
	Ding Ming, Zhang Shuling, Zhang Chen. Modeling and Testing Methods of Business Process[J]. Journal of Xi'an Jiaotong University, 2016, 50(3): 127-133.
31	Böhmer Kristof, Rinderle-Ma S. Automatic Business Process Test Case Selection: Coverage Metrics, Algorithms, and Performance Optimizations[J]. International Journal of Cooperative Information Systems, 2016, 25(4): 1740002.
32	Oditis I, Bicevskis J. Asynchronous Runtime Verification of Business Processes[C]//2015 7th International Conference on Computational Intelligence, Communication Systems and Networks. Piscataway, NJ, USA: IEEE, 2015: 103-108.
33	International Organization for Standardization. Software and Systems Engineering-software Testing-part 5: Keyword-driven Testing: [S]. Vernier, Geneva, Switzerland: International Organization for Standardization, 2016.
34	He Zhonghai, Zhang Xiang, Zhu Xiangyin. Design and Implementation of Automation Testing Framework Based on Keyword Driven[J]. Applied Mechanics and Materials, 2014, 602-605: 2142-2146.

业务模块	交换数据组数	交换数据
业务模块	交换数据组数	指令数	激励数
BM1	0	0	0
BM2	1	1	2
BM3	n	n	n
BM4	2	2	3
BM5	0	0	0
BM6	1	1	1
BM7	1	1	2
BM8	0	0	0
BM9	1	1	2
BM10	1	1	1
BM11	0	0	0

No.	流程入口	BM1	BM2	BM3	TC1	BM4	BM5	BM6	BM7	BM8	TC2	TC3	BM9	BM10	BM11	流程出口
1	单击“开始”按钮	S	F	―	―	―	―	―	―	―	―	―	―	―	―	终止
2		S	S	F	―	―	―	―	―	―	―	―	―	―	―	终止
3		S	S	F	否	―	―	―	―	―	―	―	―	―	―	终止
4		S	S	S	是	F	―	―	―	―	―	―	―	―	―	终止
5		S	S	S	是	S	S	F	F	―	―	―	―	―	―	终止
6		S	S	S	是	S	S	F	S	S	是	―	S	S	S	结束
7		S	S	S	是	S	S	F	S	S	否	是	F	―	―	终止
8		S	S	S	是	S	S	F	S	S	否	是	S	F	S	结束
9		S	S	S	是	S	S	F	S	S	否	是	S	S	S	结束
10		S	S	S	是	S	S	S	F	―	―	―	―	―	―	终止
11		S	S	S	是	S	S	S	S	S	是	―	S	S	S	结束
12		S	S	S	是	S	S	S	S	S	否	是	F	―	―	终止
13		S	S	S	是	S	S	S	S	S	否	是	S	F	S	结束
14		S	S	S	是	S	S	S	S	S	否	是	S	S	S	结束

No.	问题类别	问题级别	问题描述
1	设计缺陷	严重	设计时没有考虑信号触发、时间等能够结束流程的约束
2	设计缺陷	严重	没有设计时序参数和流程之间的约束
3	代码缺陷	严重	处理多包指令时，软件只判断了第一包数据的有效性
4	设计缺陷	严重	处理数字信号时，软件未过滤毛刺信号
5	代码缺陷	严重	第n包参数下载失败时，软件只重传一次参数下载指令
6	设计缺陷	一般	当信号或者动作异常时，主控软件没有设计流程轨迹灯的相应显示状态

[1]	赖启平, 肖谭南, 李东晟, 沈沉. 基于微分神经网络的风电机群低电压穿越特性建模[J]. 系统仿真学报, 2022, 34(12): 2546-2556.
[2]	徐鹏, 冯国奇, 代学武, 崔东亮, 魏七龙, 李宝旭, 李建明. 高铁限速运行时分的小数据驱动建模与仿真[J]. 系统仿真学报, 2021, 33(8): 1892-1904.
[3]	曾贲, 房霄, 孔德帅, 宋祥祥, 贾政轩, 林廷宇. 一种数据驱动的对抗博弈智能体建模方法[J]. 系统仿真学报, 2021, 33(12): 2838-2845.
[4]	王源源, 段雨昕, 宋光照. 装备软件可配置仿真集成技术研究[J]. 系统仿真学报, 2020, 32(11): 2093-2099.
[5]	王杰, 陈彬, 袁鹏, 马亮, 邱晓刚. 数据驱动的最优互惠避碰模型偏好速度研究[J]. 系统仿真学报, 2019, 31(12): 2731-2739.
[6]	葛渊峥, 宋智超, 孟荣清. 人工社会大规模人工人口生成方法综述[J]. 系统仿真学报, 2019, 31(10): 1951-1959.
[7]	马天雨, 王雅琳, 沈坤, 刘金平. 铝土矿连续球磨过程粒度分布预测模型[J]. 系统仿真学报, 2018, 30(2): 414-421.
[8]	陆营波, 钱晓超, 陈伟, 何舒, 陆志沣. 数据驱动的装备效能评估模型构建方法研究[J]. 系统仿真学报, 2018, 30(12): 4587-4595.
[9]	杨兴, 吴静, 周建国, 江昊, 朱劼. 基于仿真数据驱动的空间信息网络建模方法[J]. 系统仿真学报, 2018, 30(11): 4323-4334.
[10]	王戎骁, 陈彬, 邱思航, 朱正秋, 邱晓刚. 基于源项估计的污染气体扩散数据驱动仿真[J]. 系统仿真学报, 2017, 29(9): 2100-2108.
[11]	黄颖, 李康顺, 李伟, 何克清. 一种基于本体语言的业务流程建模算法[J]. 系统仿真学报, 2017, 29(10): 2282-2290.