一种数据驱动的对抗博弈智能体建模方法

doi:10.16182/j.issn1004731x.joss.20-FZ0532

系统仿真学报 ›› 2021, Vol. 33 ›› Issue (12): 2838-2845.doi: 10.16182/j.issn1004731x.joss.20-FZ0532

一种数据驱动的对抗博弈智能体建模方法

曾贲^1,2, 房霄¹, 孔德帅³, 宋祥祥¹, 贾政轩^1,2, 林廷宇^1,2

1.北京仿真中心,北京 100854;
2.北京电子工程总体研究所,北京 100854;
3.中国航天科工集团有限公司,北京 100048

收稿日期:2020-04-01 修回日期:2021-06-08 出版日期:2021-12-18 发布日期:2022-01-13
作者简介:曾贲(1994-),男,硕士,工程师,研究方向为深度强化学习技术。E-mail：duanting18@nudt.edu.cn
基金资助:
国防基础科研(JCKY2018204C004)

A Data-Driven Modeling Method for Game Adversity Agent

Zeng Bi^1,2, Fang Xiao¹, Kong Deshuai³, Song Xiangxiang¹, Jia Zhengxuan^1,2, Lin Tingyu^1,2

1. Beijing Simulation Center, Beijing 100854, China;
2. Beijing Institute of Electronic System, Beijing 100854, China;
3. China Aerospace Science and Industry Corporation Limited, Beijing 100048, China

Received:2020-04-01 Revised:2021-06-08 Online:2021-12-18 Published:2022-01-13

摘要/Abstract

摘要： 针对复杂对抗场景下编队行为协同建模及决策智能生成等问题,提出一种数据驱动的对抗博弈智能体建模方法,依托基于严肃游戏的复杂海上装备对空对抗模拟场景,通过基于并行对抗场景的分布式训练技术与基于灵巧目标的临机决策建模技术,结合空中目标、复杂海上装备等能力模型,实现人机混合增强的智能体建模,为后续深入开展复杂对抗场景下多目标协同建模研究提供了支撑。实验结果表明：深度强化学习算法能够为智能体灵巧策略的建模提供基础。

关键词: 深度强化学习, 数据驱动, 分布式训练, 临机决策

Abstract: Aiming at the problems of collaborative modeling of formation behavior and intelligent generation of decision-making in complex confrontation scenarios, based on the serious game to simulate the confrontation scenarios of complex maritime equipment against the air, this paper proposes a data-driven modeling method for game agent and uses a distributed modeling technology of parallel adversarial scenarios and opportunistic decision making technology of smart targets to achieve agent modeling. It provides support for the further exploration of multi-objective collaborative modeling in complex confrontation scenarios. The simulation results show that deep reinforcement learning algorithms can provide a basis for the modeling of agents dexterous strategies.

Key words: deep reinforcement learning, data-driven, distributed training, opportunistic decision making

中图分类号:

TP391.9

曾贲, 房霄, 孔德帅, 宋祥祥, 贾政轩, 林廷宇. 一种数据驱动的对抗博弈智能体建模方法[J]. 系统仿真学报, 2021, 33(12): 2838-2845.

Zeng Bi, Fang Xiao, Kong Deshuai, Song Xiangxiang, Jia Zhengxuan, Lin Tingyu. A Data-Driven Modeling Method for Game Adversity Agent[J]. Journal of System Simulation, 2021, 33(12): 2838-2845.

参考文献

[1] 钟华. 贴近实战的外军军事训练[J]. 国防科技, 2014, 35(4): 104-106.
Zhong Hua.Close to Actual Combat Military Training of Foreign Troops[J]. National Defense Science & Technology, 2014, 35(4): 104-106.
[2] 寇英信, 李战武, 李俊兵, 等. 现代战斗机作战任务管理与决策[M]. 北京: 国防工业出版社, 2017.
Kou Yingwin, Li Zhanwu, Li Junbing, et al.Modern Fighter Combat Mission Management and Decision-making[M]. Beijing: National Defense Industry Press, 2017.
[3] Poli R, Kennedy J, Blackwell T.Particle Swarm Optimization: An Overview[J]. Swarm Intelligence (S1935-3820), 2007(1): 33-57.
[4] Mnih V, Kavukcuoglu K, Silver D, et al.Human-level Control Through Deep Reinforcement Learning[J]. Nature (S1476-4687), 2015, 518(7540): 529-533.
[5] Silver D, Huang A, Maddison C J, et al.Mastering the Game of Go with Deep Neural Networks and Tree Search[J]. Nature (S1476-4687), 2016, 529(7587): 484-489.
[6] Vinyals O, Babuschkin I, Czarnecki W M, et al.Grandmaster Level in StarCraft II Using Multi-agent Reinforcement Learning[J]. Nature (S1476-4687), 2019, 575(7782): 350-354.
[7] Silver D, Lever G, Heess N, et al.Deterministic Policy Gradient Algorithms[C]// International Conference on Machine Learning. PMLR, 2014: 387-395.
[8] Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[J]. arXiv preprint arXiv:1412.6980, 2014.
[9] Schulman J, Levine S, Abbeel P, et al.Trust Region Policy Optimization[C]// International Conference on Machine Learning. PMLR, 2015: 1889-1897.
[10] Lillicrap T P, Hunt J J, Pritzel A, et al. Continuous Control with Deep Reinforcement Learning[J]. arXiv preprint arXiv:1509.02971, 2015.
[11] Tesauro G.Temporal Difference Learning and TD-Gammon[J]. Communications of the ACM (S0001-0782), 1995, 38(3): 58-68.
[12] Bellemare M G, Dabney W, Munos R.A Distributional Perspective on Reinforcement Learning[C]// International Conference on Machine Learning. PMLR, 2017: 449-458.
[13] Barth-Maron G, Hoffman M W, Budden D, et al. Distributed Distributional Deterministic Policy Gradients[J]. arXiv preprint arXiv:1804.08617, 2018.
[14] Sergeev A, Del Balso M. Horovod: Fast and Easy Distributed Deep Learning in TensorFlow[J]. arXiv preprint arXiv:1802.05799, 2018.

一种数据驱动的对抗博弈智能体建模方法

A Data-Driven Modeling Method for Game Adversity Agent

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 12

编辑推荐

Metrics

本文评价

[1]	张森, 张孟炎, 邵敬平, 普杰信. 基于随机策略搜索的多机三维路径规划方法[J]. 系统仿真学报, 2022, 34(6): 1286-1295.
[2]	倪凌佳, 黄晓霞, 李红旮, 张子博. 基于协作式深度强化学习的火灾应急疏散仿真研究[J]. 系统仿真学报, 2022, 34(6): 1353-1366.
[3]	王红微, 杨鹏. 基于深度强化学习的机场货运业务优化研究[J]. 系统仿真学报, 2022, 34(3): 651-660.
[4]	李启锐, 彭心怡. 基于深度强化学习的云作业调度及仿真研究[J]. 系统仿真学报, 2022, 34(2): 258-268.
[5]	徐鹏, 冯国奇, 代学武, 崔东亮, 魏七龙, 李宝旭, 李建明. 高铁限速运行时分的小数据驱动建模与仿真[J]. 系统仿真学报, 2021, 33(8): 1892-1904.
[6]	高昂, 董志明, 张国辉, 梁涛, 郭齐胜. LVC训练系统中计算机生成兵力生成技术研究[J]. 系统仿真学报, 2021, 33(3): 745-752.
[7]	王杰, 陈彬, 袁鹏, 马亮, 邱晓刚. 数据驱动的最优互惠避碰模型偏好速度研究[J]. 系统仿真学报, 2019, 31(12): 2731-2739.
[8]	葛渊峥, 宋智超, 孟荣清. 人工社会大规模人工人口生成方法综述[J]. 系统仿真学报, 2019, 31(10): 1951-1959.
[9]	马天雨, 王雅琳, 沈坤, 刘金平. 铝土矿连续球磨过程粒度分布预测模型[J]. 系统仿真学报, 2018, 30(2): 414-421.
[10]	陆营波, 钱晓超, 陈伟, 何舒, 陆志沣. 数据驱动的装备效能评估模型构建方法研究[J]. 系统仿真学报, 2018, 30(12): 4587-4595.
[11]	杨兴, 吴静, 周建国, 江昊, 朱劼. 基于仿真数据驱动的空间信息网络建模方法[J]. 系统仿真学报, 2018, 30(11): 4323-4334.
[12]	王戎骁, 陈彬, 邱思航, 朱正秋, 邱晓刚. 基于源项估计的污染气体扩散数据驱动仿真[J]. 系统仿真学报, 2017, 29(9): 2100-2108.