无人集群博弈对抗系统仿真验证及决策关键技术综述

doi:10.16182/j.issn1004731x.joss.23-0072

摘要/Abstract

摘要：

无人集群博弈对抗是一种新兴的作战样式，在智能化战争扮演着至关重要的作用，其核心是自主生成博弈对抗决策序列，为集群“赋能”。分析了无人集群博弈对抗系统仿真验证的进展；从基于专家系统和博弈论的技术、基于群体智能和优化理论的技术，以及基于神经网络和强化学习的技术三个方面论述了自主决策关键技术，以及课题组在自主决策上开展的相关工作；提出了无人集群博弈对抗的发展方向。

关键词: 无人集群, 博弈对抗, 自主决策, 系统仿真验证

Abstract:

Unmanned swarm game confrontation is a new combat mode and plays a crucial role in intelligent warfare. Its core is the autonomous generation of a series of game confrontation decision sequences to "empower" the swarm. The progress of system simulation verification for the unmanned swarm game confrontation is analyzed. The key technologies of the autonomous decision-making are discussed from three aspects, technology based on expert systems and game theory, technology based on swarm intelligence and optimization theory, and technology based on neural networks and reinforcement learning. The key technology research conducted by the author's team on the autonomous decision-making is presented. The development directions of the unmanned swarm game confrontation are proposed.

Key words: unmanned swarm, game confrontation, autonomous decision-making, system simulation validation

中图分类号:

梁晓龙,杨爱武,张佳强等 . 无人集群博弈对抗系统仿真验证及决策关键技术综述[J]. 系统仿真学报, 2024, 36(4): 805-816.

Liang Xiaolong,Yang Aiwu,Zhang Jiaqiang,et al . Simulation Verification and Decision-making Key Technologies of Unmanned Swarm Game Confrontation: A Survey[J]. Journal of System Simulation, 2024, 36(4): 805-816.

图/表 9

图1

图2

图3

图4

图5

图6

图7

表1

图8

参考文献 35

1	梁晓龙, 胡利平, 张佳强, 等. 航空集群自主空战研究进展[J]. 科技导报, 2020, 38(15): 74-88.
	Liang Xiaolong, Hu Liping, Zhang Jiaqiang, et al. Research Status and Prospect of Aircraft Swarms Autonomou Air Combat[J]. Science & Technology Review, 2020, 38(15): 74-88.
2	梁晓龙, 何吕龙, 张佳强, 等. 航空集群构型控制及其演化方法[J]. 中国科学(技术科学), 2019, 49(3): 277-287.
	Liang Xiaolong, He Lülong, Zhang Jiaqiang, et al. Configuration Control and Evolutionary Mechanism of Aircraft Swarm[J]. Scientia Sinica(Technologica), 2019, 49(3): 277-287.
3	Zhu Xiaoning. Analysis of Military Application of UAV Swarm Technology[C]//2020 3rd International Conference on Unmanned Systems (ICUS). Piscataway, NJ, USA: IEEE, 2020: 1200-1204.
4	刘雷, 刘大卫, 王晓光, 等. 无人机集群与反无人机集群发展现状及展望[J]. 航空学报, 2022, 43(增1): 4-20.
	Liu Lei, Liu Dawei, Wang Xiaoguang, et al. Development Status and Outlook of UAV Clusters and Anti-UAV Clusters[J]. Acta Aeronautica et Astronautica Sinica, 2022, 43(S1): 4-20.
5	Gulden T R, Lamb J, Hagen J, et al. Modeling Rapidly Composable, Heterogeneous, and Fractionated Forces: Findings on Mosaic Warfare from an Agent-based Model[EB/OL]. [2023-03-20]. .
6	O'Donoughue N A, McBirney S, Persons B. Distributed Kill Chains: Drawing Insights for Mosaic Warfare from the Immune System and from the Navy[EB/OL]. [2023-03-20]. .
7	Pope A P, Ide J S, Mićović Daria, et al. Hierarchical Reinforcement Learning for Air-to-air Combat[C]//2021 International Conference on Unmanned Aircraft Systems (ICUAS). Piscataway, NJ, USA: IEEE, 2021: 275-284.
8	周光霞, 周方. 美军人工智能空战系统阿尔法初探[C]//第六届中国指挥控制大会论文集(上册). 北京: 电子工业出版社, 2018: 66-70.
9	Ernest N, Carroll D, Schumacher C, et al. Genetic Fuzzy Based Artificial Intelligence for Unmanned Combat Aerial Vehicle Control in Simulated Air Combat Missions[J]. Journal of Defense Management, 2016, 6(1): 1000144.
10	Ernest N, Cohen K, Kivelevitch E, et al. Genetic Fuzzy Trees and Their Application Towards Autonomous Training and Control of a Squadron of Unmanned Combat Aerial Vehicles[J]. Unmanned Systems, 2015, 3(3): 185-204.
11	Fan D D, Theodorou E A, Reeder J. Model-based Stochastic Search for Large Scale Optimization of Multi-agent UAV Swarms[C]//2018 IEEE Symposium Series on Computational Intelligence (SSCI). Piscataway, NJ, USA: IEEE, 2018: 2216-2222.
12	Ma Yingying, Wang Guoqiang, Hu Xiaoxuan, et al. Cooperative Occupancy Decision Making of Multi-UAV in Beyond-visual-range Air Combat: A Game Theory Approach[J]. IEEE Access, 2020, 8: 11624-11634.
13	Li Shouyi, Chen Mou, Wang Yuhui, et al. Air Combat Decision-making of Multiple UCAVs Based on Constraint Strategy Games[J]. Defence Technology, 2022, 18(3): 368-383.
14	Ji Xiang, Zhang Wanpeng, Xiang Fengtao, et al. A Swarm Confrontation Method Based on Lanchester Law and Nash Equilibrium[J]. Electronics, 2022, 11(6): 896.
15	Liu Fei, Dong Xiwang, Yu Jianglong, et al. Distributed Nash Equilibrium Seeking of N-coalition Noncooperative Games with Application to UAV Swarms[J]. IEEE Transactions on Network Science and Engineering, 2022, 9(4): 2392-2405.
16	Hu Wenfeng, Zhang Qiang, Mao Yaqin. Component-Based Hierarchical State Machine-A Reusable and Flexible Game AI Technology[C]//2011 6th IEEE Joint International Information Technology and Artificial Intelligence Conference. Piscataway, NJ, USA: IEEE, 2011: 319-324.
17	Vannucchi Claudia, Diletta Romana Cacciagrano, Corradini Flavio, et al. A Formal Model for Event-condition-action Rules in Intelligent Environments[C]//12th International Conference on Intelligent Environments-5th International Workshop on the Reliability of Intelligent Environments (WoRIE 2016). Amsterdam, Netherlands: IOS Press, 2016: 56-65.
18	Duan Haibin, Li Pei, Yu Yaxiang. A Predator-prey Particle Swarm Optimization Approach to Multiple UCAV Air Combat Modeled by Dynamic Game Theory[J]. IEEE/CAA Journal of Automatica Sinica, 2015, 2(1): 11-18.
19	Gao Chunqing, Kou Yingxin, Li You, et al. Multi-objective Weapon Target Assignment Based on D-NSGA-III-A[J]. IEEE Access, 2019, 7: 50240-50254.
20	Xing Dongjing, Zhen Ziyang, Gong Huajun. Offense-defense Confrontation Decision Making for Dynamic UAV Swarm Versus UAV Swarm[J]. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, 2019, 233(15): 5689-5702.
21	Zhen Ziyang, Wen Liangdong, Wang Bolan, et al. Improved Contract Network Protocol Algorithm Based Cooperative Target Allocation of Heterogeneous UAV Swarm[J]. Aerospace Science and Technology, 2021, 119: 107054.
22	Wang Chengcai, Wu Ao, Hou Yueqi, et al. Optimal Deployment of Swarm Positions in Cooperative Interception of Multiple UAV Swarms[J]. Digital Communications and Networks, 2023, 9(2): 567-579.
23	丁超, 魏瑞轩, 周凯. 基于时域映射的多无人机系统给定时间分布式最优集结[J]. 北京航空航天大学学报, 2021, 47(2): 315-322.
	Ding Chao, Wei Ruixuan, Zhou Kai. Distributed Optimal Rendezvous of Multi-UAV Systems in Prescribed Time Based on Time-domain Mapping[J]. Journal of Beijing University of Aeronautics and Astronautics, 2021, 47(2): 315-322.
24	Hou Teng Teck, Hwee Tan Ah, Yuan Sin Tan, et al. Self-organizing Neural Networks for Learning Air Combat Maneuvers[C]//The 2012 International Joint Conference on Neural Networks (IJCNN). Piscataway, NJ, USA: IEEE, 2012: 1-8.
25	Wang Luhe, Hu Jinwen, Xu Zhao, et al. Autonomous Maneuver Strategy of Swarm Air Combat Based on DDPG[J]. Autonomous Intelligent Systems, 2021, 1(1): 15.
26	Hu Jinwen, Wang Luhe, Hu Tianmi, et al. Autonomous Maneuver Decision Making of Dual-UAV Cooperative Air Combat Based on Deep Reinforcement Learning[J]. Electronics, 2022, 11(3): 467.
27	Zhang Jiandong, Yang Qiming, Shi Guoqing, et al. UAV Cooperative Air Combat Maneuver Decision Based on Multi-agent Reinforcement Learning[J]. Journal of Systems Engineering and Electronics, 2021, 32(6): 1421-1438.
28	Xu Dan, Chen Gang. The Research on Intelligent Cooperative Combat of UAV Cluster with Multi-agent Reinforcement Learning[J]. Aerospace Systems, 2022, 5(1): 107-121.
29	Xuan Shuzhe, Ke Liangjun. UAV Swarm Attack-defense Confrontation Based on Multi-agent Reinforcement Learning[C]//Advances in Guidance, Navigation and Control. Singapore: Springer Singapore, 2022: 5599-5608.
30	Wang Baolai, Li Shengang, Gao Xianzhong, et al. Weighted Mean Field Reinforcement Learning for Large-scale UAV Swarm Confrontation[J]. Applied Intelligence, 2023, 53(5): 5274-5289.
31	Xiang Lei, Xie Tao. Research on UAV Swarm Confrontation Task Based on MADDPG Algorithm[C]//2020 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE). Piscataway, NJ, USA: IEEE, 2020: 1513-1518.
32	Källström Johan, Heintz Fredrik. Agent Coordination in Air Combat Simulation Using Multi-agent Deep Reinforcement Learning[C]//2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Piscataway, NJ, USA: IEEE, 2020: 2157-2164.
33	Wang Baolai, Li Shengang, Gao Xianzhong, et al. UAV Swarm Confrontation Using Hierarchical Multiagent Reinforcement Learning[J]. International Journal of Aerospace Engineering, 2021, 2021: 3360116.
34	Narvekar S, Peng Bei, Leonetti M, et al. Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey[J]. The Journal of Machine Learning Research, 2020, 21(1): 181.
35	Zhu Zhuangdi, Lin Kaixiang, Jain A K, et al. Transfer Learning in Deep Reinforcement Learning: A Survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(11): 13344-13362.

存在的问题	文献
训练效率低，如何引入专家知识来提高训练效率	[11, 25-27, 31]
作战目标属于稀疏奖励或延迟奖励问题	[26-28, 33]
状态空间和动作空间维度大，探索采样效率低	[30, 32-33]
策略网络泛化性、鲁棒性不足	[30, 32]
局部观测信息下作战决策	[27-29]

[1]	王积鹏, 张兴, 吴浩, 谷雨, 杨慧杰. 无人集群智能协同仿真试验评估理论框架研究[J]. 系统仿真学报, 2024, 36(4): 795-804.
[2]	王宁, 梁晓龙, 张佳强, 侯岳奇, 杨爱武. 跨域无人集群协同反潜搜索方法研究[J]. 系统仿真学报, 2024, 36(4): 817-824.
[3]	杨慧杰, 肖桃顺, 武晨, 郭凌峰. 无人集群虚实混合仿真试验环境集成构建研究[J]. 系统仿真学报, 2024, 36(4): 825-833.
[4]	郭力强, 马亮, 张会, 杨静, 范学满, 程卓. 基于博弈对抗的鱼雷抗干扰攻击建模与优化方法研究[J]. 系统仿真学报, 2023, 35(8): 1814-1823.
[5]	周鑫, 王维平, 朱一凡, 王涛, 井田. 基于人机协作的无人集群搜索方法研究[J]. 系统仿真学报, 2022, 34(4): 735-744.
[6]	赵毓, 郭继峰, 颜鹏, 白成超. 稀疏奖励下多航天器规避决策自学习仿真[J]. 系统仿真学报, 2021, 33(8): 1766-1774.
[7]	华翔, 石成泷, 李宝华, 张杰韬, 左嘉娴. 一种自适应的无人集群中心节点选择方法[J]. 系统仿真学报, 2021, 33(11): 2636-2646.