Behavior Modeling for Computer Generated Forces Based on Machine Learning

doi:10.16182/j.issn1004731x.joss.20-0931

Abstract

Abstract: With the rapid development of Machine Learning, especially deep learning, it has become an important way of modeling Computer Generated Force (CGF) behavior by ML methods, which can overcome the challenges of traditional methods. The existing research and application of three typical learning methods in CGF behavior modeling are discussed, and the effects of introducing learning into different stages of the typical CGF applications are analyzed, and the function and performance requirements of CGF behavior modeling using machine learning are proposed. Four potential research directions in the field for future are proposed.

Key words: M&S, computer generated forces, behavior modeling, machine learning, status, trends

CLC Number:

TP391.9

Zhang Qi, Zeng Junjie, Xu Kai, Qin Long, Yin Quanjun. Behavior Modeling for Computer Generated Forces Based on Machine Learning[J]. Journal of System Simulation, 2021, 33(2): 280-287.

References

[1] 郭齐胜. 计算机生成兵力导论[M]. 北京: 国防工业出版社, 2006.
Guo Qisheng.An Introduction of Computer Generated Forces[M]. Beijing: National Defence Industry Press, 2006.
[2] 黄柯棣, 刘宝宏, 黄健, 等. 作战仿真技术综述[J]. 系统仿真学报, 2004, 16(9):1887-1895.
Huang Kedi, Liu Baohong, Huang Jian, et al.A Survey of Military Simulation Technologies[J]. Journal of System Simulation, 2004, 16(9):1887-1895.
[3] 张琪. 学习驱动的CGF决策行为建模方法研究[D]. 长沙: 国防科技大学, 2019.
Zhang Qi.Learning Driven Behavior Modeling Methods for Decision Making of CGFs [D]. Changsha: National University of Defense Technology, 2019.
[4] Toubman A, Poppinga G, Roessingh J J, et al.Modeling CGF Behavior with Machine Learning Techniques: Requirements and Future Directions[C]. 2015 Interservice/Industry Training, Simulation, and Education Conference. Orlando, USA: I/ITSEC, 2015: 2637-2647.
[5] 高昂, 段莉, 张国辉, 等. 计算机生成兵力行为建模发展现状[J]. 计算机工程与应用, 2019, 55(19): 43-51.
Gao Ang, Duan Li, Zhang Guohui, et al.Development Status of Computer Generated Force Behavior Modeling[J]. Computer Engineering and Applications, 2019, 55(19): 43-51.
[6] 周志华. 机器学习[M]. 北京: 清华大学出版社, 2016.
Zhou Zhihua.Machine Learning[M]. Beijing: Tsinghua University Press, 2016.
[7] Taylor A.Designing Controllers for Computer Generated Forces with Evolutionary Computing: Experiments in a Simple Synthetic Environment[R]. Ottawa Canada: DRDC, 2013.
[8] Luotsinen L J, Kamrani F, Hammar P, et al.Evolved Creative Intelligence for Computer Generated Forces[C]. Systems, Man, and Cybernetics (SMC), 2016 IEEE International Conference on. Budapest: IEEE, 2016: 003063-003070.
[9] Ernest N D.Genetic Fuzzy Trees for Intelligent Control of Unmanned Combat Aerial Vehicles [D]. USA: Dissertations & Thesis - Gradworks, 2015.
[10] 姚剑. 面向装备效能仿真的战术探索方法研究[D]. 长沙: 国防科技大学, 2018.
Yao Jian.Study on Tactics Exploration Method for Equipment Effectiveness Simulation[D]. Changsha: National University of Defense Technology, 2018.
[11] 高阳, 陈世福, 陆鑫. 强化学习研究综述[J]. 自动化学报, 2004, 30(1): 86-100.
Gao Yang, Chen Shifu, Lu Xin.Research on Reinforcement Learning Technology:A Review[J]. Automatica Sinica, 2004, 30(1):86-100.
[12] Aihe D O, Gonzalez A J.Correcting Flawed Expert Knowledge Through Reinforcement Learning[J]. Expert Systems with Applications (S0957-4174), 2015, 42(17): 6457-6471.
[13] Teng T H, Tan A H, Teow L N.Adaptive Computer Generated Forces for Simulator-based Training[J]. Expert Systems with Applications (S0957-4174), 2013, 40(18): 7341-7353.
[14] Junges R, Klugl F.Modeling Agent Behavior Through Online Evolutionary and Reinforcement Learning[C]. Federated Conference on Computer Science and Information Systems. USA: IEEE, 2011: 643-650.
[15] Spronck P, Ponsen M, Sprink Huizen-Kuyper I, et al. Adaptive Game AI with Dynamic Scripting[J]. Machine Learning (S0885-6125), 2006, 63(3): 217-248.
[16] Toubman A.Calculated Moves: Generating Air Combat Behaviour[D]. Netherlands: Dissertations & Thesis, Leiden University, 2020.
[17] Toubman A, Roessingh J J M, Spronck P, et al. Improving Air-to-Air Combat Behavior Through Transparent Machine Learning[C]. Interservice/Industry Training, Simulation, and Education Conference. Orlando, USA: I/ITSEC, 2014: 1-11.
[18] Fernlund H K G, Gonzalez A J, Georgiopoulos M, et al. Learning Tactical Human Behavior Through Observation of Human Performance[J]. IEEE Transactions on Systems Man & Cybernetics Part B Cybernetics (S1083-4419), 2006, 36(1): 128.
[19] Argall B D, Chernova S, Veloso M, et al.A Survey of Robot Learning from Demonstration[J]. Robotics & Autonomous Systems (S0921-8890), 2009, 57(5): 469-483.
[20] Moriarty C L, Gonzalez A J.Learning Human Behavior from Observation for Gaming Applications[C]. International Florida Artificial Intelligence Research Society Conference, May 19-21, 2009. Sanibel Island, Florida, USA: DBLP, 2009.
[21] Johnson C L, Avelino J G.Learning Collaborative Team Behavior from Observation[J]. Expert Systems with Applications (S0957-4174), 2014, 41(5): 2316-2328.
[22] Berthling-Hansen G, Morch E, Lovlid R A, et al.Automating Behavior Tree Generation for Simulating Troop Movements (Poster)[C]. 2018 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA). USA: IEEE, 2018: 147-153.
[23] Floyd M W.A General-Purpose Framework for Learning by Observation[D]. Ottawa, Ontario, Canada: University of Carleton, 2013.
[24] Ontan S, Mishra K, Sugandh N, et al.Case-Based Planning and Execution for Real-Time Strategy Games[C]. International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development. UK: Springer-Verlag, 2007: 164-178.
[25] Robertson G, Watson I.Building Behavior Trees from Observations in Real-time Strategy Games[C]. International Symposium on Innovations in Intelligent Systems and Applications. Madrid: IEEE, 2015: 1-7.
[26] Goodfellow I, Pouget-Abadie J, Mirza M, et al.Generative Adversarial Nets[C]. NIPS. USA: MIT, 2014: 2672-2680.
[27] Jonathan H, Stefano E.Generative adversarial Imitation Learning[C]. Advances in Neural Information Processing Systems. New York, NY: Curran Associates, 2016: 4565-4573.
[28] Stein G, Gonzalez A J.Building High-Performing Human-Like Tactical Agents Through Observation and Experience[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B Cybernetics (S1083-4419), 2011, 41(3):792-804.
[29] Luotsinen L J, Lovlid R A.Data-driven Behavior Modeling for Computer Generated Forces[C]. NATO Modeling and Simulation Group Symp. M&S Support to Operational Tasks including War-gaming, Logistics, Cyber Defence (MSG-133). Germany: NATO, 2015: 1-13.
[30] Kamrani F, Luotsinen L J, Lovlid R A.Learning Objective Agent Behavior using a Data-driven Modeling Approach[C]. IEEE International Conference on Systems, Man, and Cybernetics. Budapest: IEEE, 2016: 002175-002181.
[31] Petty M D.Benefits and Consequences of Automated Learning in Computer Generated Forces Systems[J]. Information & Security: An International Journal (S0861-5160), 2003, 12(1): 63-74.
[32] Babak Toghiani-Rizi.Evaluation of Deep Learning Methods for Creating Synthetic Actors[D]. Netherlands: Dissertations & Thesis, 2017.
[33] Oana-Maria Camburu.Explaining Deep Neural Networks[D]. Oxford, England: Dissertations & Thesis, University of Oxford, 2020.