Research on CGF-oriented Natural Language Interaction Framework

doi:10.16182/j.issn1004731x.joss.24-0921

Abstract

Abstract:

To address the mismatch between existing natural language interaction frameworks and training tasks in simulation-based military training, which limits smooth interaction between trainees and Computer Generated Forces (CGF), this paper proposes a Natural Language Interaction framework for Computer Generated Forces (NLI4CGF). The framework analyzes the logic and functional requirements of natural language interaction between trainees and CGF, and establishes an interaction architecture tailored for military simulation training scenarios. It supports semantic parsing and knowledge query tasks within a prototype system developed for infantry squad simulation training. Experimental results demonstrate that the proposed model performs effectively, meets the requirements of infantry training, and significantly enhances the trainee's interactive experience.

Key words: simulation-based military training, computer generated forces(CGF), natural language interaction, natural language processing, semantic parsing, knowledge querying

CLC Number:

TP391.9

Li Xinmeng, Xu Kai, Hu Yue, Huang Hesong, Yin Quanjun. Research on CGF-oriented Natural Language Interaction Framework[J]. Journal of System Simulation, 2025, 37(11): 2853-2866.

Figures/Tables 15

Fig. 1

Table 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Table 2

Fig. 8

Fig. 9

Fig. 10

Table 3

Table 4

Fig. 11

References 28

[1]	高昂, 董志明, 张国辉, 等. LVC训练系统中计算机生成兵力生成技术研究[J]. 系统仿真学报, 2021, 33(3): 745-752.
	Gao Ang, Dong Zhiming, Zhang Guohui, et al. Research on Generation Technology of Computer Generated Force in LVC Training System[J]. Journal of System Simulation, 2021, 33(3): 745-752.
[2]	尹全军. 基于多Agent的计算机生成兵力建模与仿真[D]. 长沙: 国防科学技术大学, 2005.
	Yin Quanjun. Modeling and Simulation of Computer Generated Forces Based on Multi-agent[D]. Changsha: National University of Defense Technology, 2005.
[3]	周楠, 艾剑良. 基于HMM和RNN的无人机语音控制方案与仿真研究[J]. 系统仿真学报, 2020, 32(3): 464-471.
	Zhou Nan, Ai Jianliang. Speech Control Scheme Design and Simulation for UAV Based on HMM and RNN[J]. Journal of System Simulation, 2020, 32(3): 464-471.
[4]	Ni Jinjie, Young Tom, Pandelea Vlad, et al. Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey[J]. Artificial Intelligence Review, 2023, 56(4): 3055-3155.
[5]	Ball J, Heiberg A, Silber R. Toward a Large-scale Model of Language Comprehension in Act-R 6[C]//Proceedings of the 8th International Conference on Cognitive Modeling. Ann Arbor, Michigan: Taylor & Francis, 2007: 173-179.
[6]	van Oijen Joost, Claessen Olivier. Building Conversational Agents for Military Training: Towards a Virtual Wingman[C]//Artificial Intelligence in HCI. Cham: Springer International Publishing, 2021: 516-531.
[7]	徐享忠, 汤再江, 谭亚新. 作战仿真与指控系统语义互操作技术参考框架研究[J]. 系统仿真学报, 2015, 27(8): 1735-1739.
	Xu Xiangzhong, Tang Zaijiang, Tan Yaxin. Technical Reference Framework for Semantic Interoperation Between Combat Simulation Systems and Command & Control Systems[J]. Journal of System Simulation, 2015, 27(8): 1735-1739.
[8]	赵阳洋, 王振宇, 王佩, 等. 任务型对话系统研究综述[J]. 计算机学报, 2020, 43(10): 1862-1896.
	Zhao Yangyang, Wang Zhenyu, Wang Pei, et al. A Survey on Task-oriented Dialogue Systems[J]. Chinese Journal of Computers, 2020, 43(10): 1862-1896.
[9]	Zamani H, Craswell N. Macaw: An Extensible Conversational Information Seeking Platform[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: Association for Computing Machinery, 2020: 2193-2196.
[10]	Deldjoo Yashar, Trippas J R, Zamani H. Towards Multi-modal Conversational Information Seeking[C]//Proceedings of the 44th International ACM SIGIR conference on research and development in Information Retrieval. New York: Association for Computing Machinery, 2021: 1577-1587.
[11]	Soo Chang Doo, Hee Cho Gun, Yong Suk Choi. Ontology-based Knowledge Model for Human-robot Interactive Services[C]//Proceedings of the 35th Annual ACM Symposium on Applied Computing. New York: Association for Computing Machinery, 2020: 2029-2038.
[12]	Yu Dong, Deng Li. Automatic Speech Recognition: A Deep Learning Approach[M]. London: Springer London, 2015.
[13]	Kumar Yogesh, Koul Apeksha, Singh Chamkaur. A Deep Learning Approaches in Text-to-speech System: A Systematic Review and Recent Research Perspective[J]. Multimedia Tools and Applications, 2023, 82(10): 15171-15197.
[14]	刘永红, 薛青, 曹波伟, 等. 基于指令机制的作战Agent 体系结构研究[J].计算机仿真(增1), 2011 :120-123.
	Liu Yonghong, Xue Qing, Cao Bowei, et al. Research on Architecture of Combat Agent Based-on Command Mechanism[J]. Computer Simulation(S1), 2011: 120-123.
[15]	Brabra Hayet, Báez Marcos, Benatallah Boualem, et al. Dialogue Management in Conversational Systems: A Review of Approaches, Challenges, and Opportunities[J]. IEEE Transactions on Cognitive and Developmental Systems, 2022, 14(3): 783-798.
[16]	Duan Jiafei, Yu Samson, Tan Huili, et al. A Survey of Embodied AI: from Simulators to Research Tasks[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2022, 6(2): 230-244.
[17]	Giorgi I, Cangelosi A, Masala G L. Learning Actions from Natural Language Instructions Using an ON-world Embodied Cognitive Architecture[J]. Frontiers in Neurorobotics, 2021, 15: 626380.
[18]	Harsimran Jit Singh, Bathla Gourav, Mehta Munish, et al. Visual Questions Answering Developments, Applications, Datasets and Opportunities: A State-of-the-art Survey[C]//2023 International Conference on Sustainable Computing and Data Communication Systems (ICSCDS). Piscataway: IEEE, 2023: 778-785.
[19]	蒋锴, 钱夔, 郑玄. 基于知识图谱的军事信息搜索技术架构[J]. 指挥信息系统与技术, 2016, 7(1): 47-52.
	Jiang Kai, Qian Kui, Zheng Xuan. Military Information Retrieval Technology Architecture Based on Knowledge Graph[J]. Command Information System and Technology, 2016, 7(1): 47-52.
[20]	殷来祥, 李志强, 付琼莹. 基于NL2SQL的兵棋数据智能统计分析方法研究[J]. 系统仿真学报, 2023, 35(9): 2000-2010.
	Yin Laixiang, Li Zhiqiang, Fu Qiongying. Research on Intelligent Statistical Analysis of Wargaming Data Based on NL2SQL[J]. Journal of System Simulation, 2023, 35(9): 2000-2010.
[21]	孙晓雅. 面向非结构化文档的军事领域自动问答技术研究[D]. 长沙: 国防科技大学, 2020.
	Sun Xiaoya. Research on Question Answering Technology Towards Unstructured Documents in Military Field[D]. Changsha: National University of Defense Technology, 2020.
[22]	Gao Shuyang, Sethi A, Agarwal S, et al. Dialog State Tracking: A Neural Reading Comprehension Approach[C]//Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue. Stroudsburg: ACL, 2019: 264-273.
[23]	Wu Chien-Sheng, Madotto Andrea, Hosseini-Asl Ehsan, et al. Transferable Multi-domain State Generator for Task-oriented Dialogue Systems[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2019: 808-819.
[24]	Li Xinmeng, Li Qian, Wu Wansen, et al. Generation and Extraction Combined Dialogue State Tracking with Hierarchical Ontology Integration[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2021: 2241-2249.
[25]	Zhou Mantong, Huang Minlie, Zhu Xiaoyan. An Interpretable Reasoning Network for Multi-relation Question Answering[C]//Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg: ACL, 2018: 2010-2022.
[26]	Saxena Apoorv, Tripathi Aditay, Talukdar Partha. Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 4498-4507.
[27]	Li Xinmeng, Alazab Mamoun, Li Qian, et al. Question-aware Memory Network for Multi-hop Question Answering in Human-robot Interaction[J]. Complex & Intelligent Systems, 2022, 8(2): 851-861.
[28]	蒋文明. 面向中文文本的空间方位关系抽取方法研究[D]. 南京: 南京师范大学, 2010.

能力	输入	输出	技术路线	难点
作战指令理解	文本指令	结构化指令	语义解析	场景扩展，上下文追踪，未登录词的识别
战术任务执行	文本指令，环境状态	机动动作序列	视觉语言导航	多模态信息的融合、对齐与推理
战术任务执行	文本指令，环境状态	操作动作序列	操作序列生成	多模态信息的融合、对齐与推理
战场信息查询	查询问题，视频/图像	答案实体	视觉/图像问答	多模态信息的融合、对齐与推理
	查询问题，知识图谱	答案实体	知识图谱问答	复杂查询问题的关系推理
	查询问题，关系型数据库	数据库查询结果	数据库自然语言接口	关系数据库多表关联查询问题的解析
	查询问题，文本文档	文本片段	机器阅读理解	多跳问题与多篇文本的理解和推理

模型	SA	SP	F1	JA
DSTReader	98.5	99.1	97.0	88.2
TRADE	98.1	99.0	97.9	86.3
GeeX	99.5	99.4	98.7	91.5

模型	WQA-1H		WQA-2H		WQA-3H
模型	HIT@1	MRR	HIT@1	MRR	HIT@1	MRR
IRN	99.5	98.9	93.4	91.5	89.5	89.0
Embed- KGQA	99.2	98.8	94.1	92.4	90.8	90.4
QA2MN	99.6	99.2	95.2	93.2	92.1	91.3

编号	准确度	便捷性	沉浸感
平均值	10.3/4.7/5	10.6/4/5.3	10.6/4.3/5
1	13/3/4	9/5/6	10/4/6
2	10/6/4	12/3/5	10/5/5
3	8/5/7	11/4/5	12/4/4