基于机器学习的计算机生成兵力行为建模研究综述

doi:10.16182/j.issn1004731x.joss.20-0931

摘要/Abstract

摘要： 随着机器学习特别是深度学习技术的快速发展,采用学习方法辅助进行军用仿真中计算机生成兵力(Computer Generated Force,CGF)的行为建模,已成为克服传统有限状态机等基于知识工程方法面临的建模效率低,自适应差等问题的重要途径和发展方向。本文对采用机器学习进行CGF行为建模的应用现状、需求及发展趋势等若干问题进行了系统论述。梳理总结了CGF行为建模中三类典型学习方法的应用现状;分析了三类典型军用仿真系统引入学习的优缺点和带来的影响;提出CGF系统对学习建模方法的功能和性能需求;提出并探讨了该领域未来四个发展趋势和重点研究方向。

关键词: 建模与仿真, 计算机生成兵力, 行为建模, 机器学习, 现状, 趋势

Abstract: With the rapid development of Machine Learning, especially deep learning, it has become an important way of modeling Computer Generated Force (CGF) behavior by ML methods, which can overcome the challenges of traditional methods. The existing research and application of three typical learning methods in CGF behavior modeling are discussed, and the effects of introducing learning into different stages of the typical CGF applications are analyzed, and the function and performance requirements of CGF behavior modeling using machine learning are proposed. Four potential research directions in the field for future are proposed.

Key words: M&S, computer generated forces, behavior modeling, machine learning, status, trends

中图分类号:

TP391.9

张琪, 曾俊杰, 许凯, 秦龙, 尹全军. 基于机器学习的计算机生成兵力行为建模研究综述[J]. 系统仿真学报, 2021, 33(2): 280-287.

Zhang Qi, Zeng Junjie, Xu Kai, Qin Long, Yin Quanjun. Behavior Modeling for Computer Generated Forces Based on Machine Learning[J]. Journal of System Simulation, 2021, 33(2): 280-287.

参考文献

[1] 郭齐胜. 计算机生成兵力导论[M]. 北京: 国防工业出版社, 2006.
Guo Qisheng.An Introduction of Computer Generated Forces[M]. Beijing: National Defence Industry Press, 2006.
[2] 黄柯棣, 刘宝宏, 黄健, 等. 作战仿真技术综述[J]. 系统仿真学报, 2004, 16(9):1887-1895.
Huang Kedi, Liu Baohong, Huang Jian, et al.A Survey of Military Simulation Technologies[J]. Journal of System Simulation, 2004, 16(9):1887-1895.
[3] 张琪. 学习驱动的CGF决策行为建模方法研究[D]. 长沙: 国防科技大学, 2019.
Zhang Qi.Learning Driven Behavior Modeling Methods for Decision Making of CGFs [D]. Changsha: National University of Defense Technology, 2019.
[4] Toubman A, Poppinga G, Roessingh J J, et al.Modeling CGF Behavior with Machine Learning Techniques: Requirements and Future Directions[C]. 2015 Interservice/Industry Training, Simulation, and Education Conference. Orlando, USA: I/ITSEC, 2015: 2637-2647.
[5] 高昂, 段莉, 张国辉, 等. 计算机生成兵力行为建模发展现状[J]. 计算机工程与应用, 2019, 55(19): 43-51.
Gao Ang, Duan Li, Zhang Guohui, et al.Development Status of Computer Generated Force Behavior Modeling[J]. Computer Engineering and Applications, 2019, 55(19): 43-51.
[6] 周志华. 机器学习[M]. 北京: 清华大学出版社, 2016.
Zhou Zhihua.Machine Learning[M]. Beijing: Tsinghua University Press, 2016.
[7] Taylor A.Designing Controllers for Computer Generated Forces with Evolutionary Computing: Experiments in a Simple Synthetic Environment[R]. Ottawa Canada: DRDC, 2013.
[8] Luotsinen L J, Kamrani F, Hammar P, et al.Evolved Creative Intelligence for Computer Generated Forces[C]. Systems, Man, and Cybernetics (SMC), 2016 IEEE International Conference on. Budapest: IEEE, 2016: 003063-003070.
[9] Ernest N D.Genetic Fuzzy Trees for Intelligent Control of Unmanned Combat Aerial Vehicles [D]. USA: Dissertations & Thesis - Gradworks, 2015.
[10] 姚剑. 面向装备效能仿真的战术探索方法研究[D]. 长沙: 国防科技大学, 2018.
Yao Jian.Study on Tactics Exploration Method for Equipment Effectiveness Simulation[D]. Changsha: National University of Defense Technology, 2018.
[11] 高阳, 陈世福, 陆鑫. 强化学习研究综述[J]. 自动化学报, 2004, 30(1): 86-100.
Gao Yang, Chen Shifu, Lu Xin.Research on Reinforcement Learning Technology:A Review[J]. Automatica Sinica, 2004, 30(1):86-100.
[12] Aihe D O, Gonzalez A J.Correcting Flawed Expert Knowledge Through Reinforcement Learning[J]. Expert Systems with Applications (S0957-4174), 2015, 42(17): 6457-6471.
[13] Teng T H, Tan A H, Teow L N.Adaptive Computer Generated Forces for Simulator-based Training[J]. Expert Systems with Applications (S0957-4174), 2013, 40(18): 7341-7353.
[14] Junges R, Klugl F.Modeling Agent Behavior Through Online Evolutionary and Reinforcement Learning[C]. Federated Conference on Computer Science and Information Systems. USA: IEEE, 2011: 643-650.
[15] Spronck P, Ponsen M, Sprink Huizen-Kuyper I, et al. Adaptive Game AI with Dynamic Scripting[J]. Machine Learning (S0885-6125), 2006, 63(3): 217-248.
[16] Toubman A.Calculated Moves: Generating Air Combat Behaviour[D]. Netherlands: Dissertations & Thesis, Leiden University, 2020.
[17] Toubman A, Roessingh J J M, Spronck P, et al. Improving Air-to-Air Combat Behavior Through Transparent Machine Learning[C]. Interservice/Industry Training, Simulation, and Education Conference. Orlando, USA: I/ITSEC, 2014: 1-11.
[18] Fernlund H K G, Gonzalez A J, Georgiopoulos M, et al. Learning Tactical Human Behavior Through Observation of Human Performance[J]. IEEE Transactions on Systems Man & Cybernetics Part B Cybernetics (S1083-4419), 2006, 36(1): 128.
[19] Argall B D, Chernova S, Veloso M, et al.A Survey of Robot Learning from Demonstration[J]. Robotics & Autonomous Systems (S0921-8890), 2009, 57(5): 469-483.
[20] Moriarty C L, Gonzalez A J.Learning Human Behavior from Observation for Gaming Applications[C]. International Florida Artificial Intelligence Research Society Conference, May 19-21, 2009. Sanibel Island, Florida, USA: DBLP, 2009.
[21] Johnson C L, Avelino J G.Learning Collaborative Team Behavior from Observation[J]. Expert Systems with Applications (S0957-4174), 2014, 41(5): 2316-2328.
[22] Berthling-Hansen G, Morch E, Lovlid R A, et al.Automating Behavior Tree Generation for Simulating Troop Movements (Poster)[C]. 2018 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA). USA: IEEE, 2018: 147-153.
[23] Floyd M W.A General-Purpose Framework for Learning by Observation[D]. Ottawa, Ontario, Canada: University of Carleton, 2013.
[24] Ontan S, Mishra K, Sugandh N, et al.Case-Based Planning and Execution for Real-Time Strategy Games[C]. International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development. UK: Springer-Verlag, 2007: 164-178.
[25] Robertson G, Watson I.Building Behavior Trees from Observations in Real-time Strategy Games[C]. International Symposium on Innovations in Intelligent Systems and Applications. Madrid: IEEE, 2015: 1-7.
[26] Goodfellow I, Pouget-Abadie J, Mirza M, et al.Generative Adversarial Nets[C]. NIPS. USA: MIT, 2014: 2672-2680.
[27] Jonathan H, Stefano E.Generative adversarial Imitation Learning[C]. Advances in Neural Information Processing Systems. New York, NY: Curran Associates, 2016: 4565-4573.
[28] Stein G, Gonzalez A J.Building High-Performing Human-Like Tactical Agents Through Observation and Experience[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B Cybernetics (S1083-4419), 2011, 41(3):792-804.
[29] Luotsinen L J, Lovlid R A.Data-driven Behavior Modeling for Computer Generated Forces[C]. NATO Modeling and Simulation Group Symp. M&S Support to Operational Tasks including War-gaming, Logistics, Cyber Defence (MSG-133). Germany: NATO, 2015: 1-13.
[30] Kamrani F, Luotsinen L J, Lovlid R A.Learning Objective Agent Behavior using a Data-driven Modeling Approach[C]. IEEE International Conference on Systems, Man, and Cybernetics. Budapest: IEEE, 2016: 002175-002181.
[31] Petty M D.Benefits and Consequences of Automated Learning in Computer Generated Forces Systems[J]. Information & Security: An International Journal (S0861-5160), 2003, 12(1): 63-74.
[32] Babak Toghiani-Rizi.Evaluation of Deep Learning Methods for Creating Synthetic Actors[D]. Netherlands: Dissertations & Thesis, 2017.
[33] Oana-Maria Camburu.Explaining Deep Neural Networks[D]. Oxford, England: Dissertations & Thesis, University of Oxford, 2020.