Spatio-Temporal Memory Models in Spatial Cognitive Behavioral Modeling

doi:10.16182/j.issn1004731x.joss.20-0936

Abstract

Abstract: It is crucial for military simulation systems to model the layouts of spatial environments and the temporal logic of entity activities. To improve the human-likeness of Computer Generated Forces (CGFs) and the runtime efficiency of the systems, building spatio-temporal memory representations and reasoning mechanisms that accord to mankind cognition is required. The neuro cognition science background of spatio-temporal memory is presented, and the related work on modeling the memory in deep learning and computational intelligence communities is summarized. Following the OODA (Observe-Orient-Decide-Act) loop, a general framework of spatial cognitive behavioral modeling is designed and the application of spatio-temporal memory in different phases including qualitative spatial representations, situation comprehension and projection, online planning and acting is proposed. The framework can guide the effective modeling of spatial representations and reasonings for CGFs.

Key words: computer generated forces, spatial cognitive behavioral modeling, spatio-temporal memory, situation awareness

CLC Number:

TP391.9

Hu Yue, Xu Kai, Qin Long, Yin Quanjun, Zha Yabing. Spatio-Temporal Memory Models in Spatial Cognitive Behavioral Modeling[J]. Journal of System Simulation, 2021, 33(2): 249-261.

References

[1] Hart R A, Moore G T.The Development of Spatial Cognition: A review[M]. New Jersey, USA: Aldine Transaction, 1973.
[2] 尹全军. 基于多Agent 的计算机生成兵力建模与仿真 [D]. 长沙: 国防科技大学, 2005.
Yin Quanjun.Modeling and Simulation of Computer-Generated Forces based on Multiagent[D]. Changsha: National University of Defense Technology, 2005.
[3] 秦龙. 面向CGF 的战场空间表示及推理关键技术研究[D]. 长沙: 国防科学技术大学, 2014.
Qin Long.Research on Key Technologies of Battlefield Spatial Representation and Reasoning of CGFs[D]. Changsha: National University of Defense Technology, 2014.
[4] Tolman E C.Cognitive Maps in Rats and Men[J]. Psychological Review (S0033-295X), 1948, 55(4): 189.
[5] Atkinson R C, Shiffrin R M.Human Memory: A Proposed System and its Control Processes[J]. Psychology of Learning and Motivation (S0079-7421). 1968, 2(4): 89-195.
[6] Baddeley A.Working memory[J]. Science (S1095-9203), 1992, 255(5044): 556-559.
[7] Baddeley A.The Episodic Buffer: a New Component of Working Memory?[J]. Trends in Cognitive Sciences (S1364-6613), 2000, 4(11): 417-423.
[8] Cowan N.What are the Differences Between Long-term, Short-term, and Working Memory?[J]. Progress in Brain Research (S0079-6123), 2008, 169: 323-338.
[9] Stickgold R.Sleep-dependent Memory Consolidation[J]. Nature (S0028-0836), 2005, 437(7063): 1272-1278.
[10] Tulving E.Episodic Memory: From Mind to Brain[J]. Annual Review of Psychology (S0066-4308), 2002, 53(1): 1-25.
[11] Wiggs C L, Weisberg J, Martin A.Neural Correlates of Semantic and Episodic Memory Retrieval[J]. Neuropsychologia (S0028-3932), 1998, 37(1): 103-118.
[12] Cohen M D, Bacdayan P.Organizational Routines Are Stored as Procedural Memory: Evidence from a Laboratory Study[J]. Organization Science (S1047-7039), 1994, 5(4): 554-568.
[13] Olton D S.Spatial Memory[J]. Scientific American (S0036-8733), 1977, 236(6): 82-99.
[14] Olton D S, Paras B C.Spatial Memory and Hippocampal Function[J]. Neuropsychologia (S0028-3932), 1979, 17(6): 669-682.
[15] Sadalla E K, Burroughs W J, Staplin L J.Reference Points in Spatial Cognition[J]. Journal of Experimental Psychology: Human Learning and Memory (S0096-1515), 1980, 6(5): 516.
[16] Foo P, Warren W H, Duchon A, et al.Do Humans Integrate Routes into a Cognitive Map? Map-versus Landmark-based Navigation of Novel Shortcuts[J]. Journal of Experimental Psychology: Learning, Memory, and Cognition (S0278-7393), 2005, 31(2): 195.
[17] Deshmukh S S, Knierim J J.Influence of Local Objects on Hippocampal Representations: Landmark Vectors and Memory[J]. Hippocampus (S1050-9631), 2013, 23(4): 253-267.
[18] O'keefe J, Nadel L. The Hippocampus as a Cognitive Map[M]. Oxford: Clarendon Press, 1978.
[19] Hafting T, Fyhn M, Molden S, et al.Microstructure of a Spatial Map in the Entorhinal Cortex[J]. Nature (S0028-0836), 2005, 436(7052): 801.
[20] Barry C, Lever C, Hayman R, et al.The Boundary Vector Cell Model of Place Cell Firing and Spatial Memory[J]. Reviews in the Neurosciences (S0334-1763), 2006, 17(1/2): 71.
[21] Gallistel C R.The Organization of Learning[M]. Cambridge, MA, USA: The MIT Press, 1990.
[22] Collett T S, Graham P.Animal Navigation: Path Integration, Visual Landmarks and Cognitive Maps[J]. Current Biology (S0960-9822), 2004, 14(12): 475.
[23] Foo P, Duchon A, Warren W H, et al.Humans do Not Switch Between Path Knowledge and Landmarks When Learning a New Environment[J]. Psychological Research (S0340-0727), 2007, 71(3): 240.
[24] Epstein R A, Patai E Z, Julian J B, et al.The Cognitive Map in Humans: Spatial Navigation and Beyond[J]. Nature Neuroscience (S1097-6256), 2017, 20(11): 1504.
[25] Eichenbaum H, Cohen N J.Can We Reconcile the Declarative Memory and Spatial Navigation Views on Hippocampal Function?[J]. Neuron (S0896-6273), 2014, 83(4): 764-770.
[26] Schiller D, Eichenbaum H, Buffalo E A, et al.Memory and Space: Towards an Understanding of the Cognitive Map[J]. Journal of Neuroscience (S0270-6474), 2015, 35(41): 13904-13911.
[27] Eichenbaum H.The Role of the Hippocampus in Navigation is Memory[J]. Journal of Neurophysiology (S0022-3077), 2017, 117(4): 1785-1796.
[28] Hochreiter S, Schmidhuber J.Long Short-term Memory[J]. Neural Computation (S0899-7667), 1997, 9(8): 1735.
[29] Merity S, Keskar N S, Socher R.Regularizing and Optimizing LSTM Language Models[EB/OL]. [2020-11-10].https://arxiv.org/abs/1708.02182.
[30] Rahmatizadeh R, Abolghasemi P, Behal A, et al.From Virtual Demonstration to Real-world Manipulation Using LSTM and MDN[EB/OL]. [2020-11-10].https://arxiv.org/abs/1603.03833.
[31] Mirowski P, Pascanu R, Viola F, et al. Learning to Navigate in Complex Environments[EB/OL]. [2020-11-10]. http://arxiv.org/abs/1611.03673.
[32] Graves A, Wayne G, Danihelka I. Neural Turing Machines[EB/OL]. [2020-11-10]. http://arxiv.org/abs/ 1410.5401.
[33] Sukhbaatar S, Weston J, Fergus R.End-to-end Memory Networks[C]. Advances in Neural Information Processing Systems. Montreal, CAN: Neural Information Processing Systems Foundation, 2015: 2440-2448.
[34] Gupta S, Davidson J, Levine S, et al.Cognitive Mapping and Planning for Visual Navigation[C]. IEEE Conference on Computer Vision and Pattern Recognition. Hawaii, USA: IEEE, 2017: 2616-2625.
[35] Zhang J, Tai L, Boedecker J, et al. Neural SLAM: Learning to Explore with External Memory[EB/OL]. [2020-11-10]. http://arxiv.org/abs/1706.09520.
[36] Gupta S, Fouhey D, Levine S, et al. Unifying Map and Landmark Based Representations for Visual Navigation [EB/OL]. [2020-11-10]. http://arxiv.org/abs/1712.08125.
[37] Parisotto E, Salakhutdinov R. Neural Map: Structured Memory for Deep Reinforcement Learning[EB/OL]. [2020-11-10]. http://arxiv.org/abs/1702.08360.
[38] Fang K, Toshev A, Fei-Fei L, et al.Scene Memory Transformer for Embodied Agents in Long-horizon Tasks[C]. IEEE Conference on Computer Vision and Pattern Recognition. CA, USA: IEEE, 2019: 538-547.
[39] Oh J, Chockalingam V, Singh S, et al. Control of Memory, Active Perception, and Action in Minecraft [EB/OL]. [2020-11-10]. http://arxiv.org/abs/1605.09128.
[40] Savinov N, Dosovitskiy A, Koltun V. Semi-parametric Topological Memory for Navigation[EB/OL]. [2020-11-10]. http://arxiv.org/abs/1803.00653.
[41] Eysenbach B, Salakhutdinov R R, Levine S.Search on the Replay Buffer: Bridging Planning and Reinforcement Learning[C]. Advances in Neural Information Processing Systems. Vancouver CAN: Neural Information Processing Systems Foundation, 2019: 15246-15257.
[42] Mueller S T, Shiffrin R M.REM II: A Model of the Developmental Co-evolution of Episodic Memory and Semantic Knowledge[C]. International conference on learning and development (ICDL). Indiana, USA: IEEE Computational Intelligence Society, 2006.
[43] Nuxoll A M, Laird J E.Extending Cognitive Architecture with Episodic Memory[C]. AAAI. Vancouver, CAN: AAAI Press, 2007: 1560-1564.
[44] Laird J E, Mohan S.A Case Study of Knowledge Integration Across Multiple Memories in SOAR[J]. Biologically Inspired Cognitive Architectures (S2212-683X), 2014, 1(8): 93-99.
[45] Shastri L.Episodic Memory and Cortico-hippocampal Interactions[J]. Trends in Cognitive Sciences (S1364-6613), 2002, 6(4): 162-168.
[46] Carpenter G A, Grossberg S.A Massively Parallel Architecture for a Self-organizing Neural Pattern Recognition Machine[J]. Computer Vision, Graphics, and Image Processing (S0734-189X), 1987, 37(1): 54-115.
[47] Wang W, Subagdja B, Tan A H, et al.Neural Modeling of Episodic Memory: Encoding, Retrieval, and Forgetting[J]. IEEE Transactions on Neural Networks and Learning Systems (S2162-237X), 2012, 23(10): 1574-1586.
[48] Chang P H, Tan A H.Encoding and Recall of Spatio-Temporal Episodic Memory in Real Time[C]. IJCAI. Melbourne, Australia: IJCAI, 2017: 1490-1496.
[49] Collins A M, Quillian M R.Retrieval Time from Semantic Memory[J]. Journal of Verbal Learning and Verbal Behavior (S0022-5371), 1969, 8(2): 240-247.
[50] Kosko B.Fuzzy Cognitive Maps[J]. International Journal of Man-machine Studies (S0020-7373), 1986, 24(1): 65-75.
[51] Tung W L, Quek C. eFSM—A Novel Online Neural-fuzzy Semantic Memory Model[J]. IEEE Transactions on Neural Networks (S1045-9227), 2009, 21(1): 136-157.
[52] Wang W, Tan A H, Teow L N.Semantic Memory Modeling and Memory Interaction in Learning Agents[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems (S2168-2216), 2016, 47(11): 2882-2895.
[53] O’Reilly R C, Bhattacharyya R, Howard M D, et al. Complementary Learning Systems[J]. Cognitive Science (S1879-307X), 2014, 38(6): 1229-1248.
[54] Sun R, Mathews R C.Implicit cognition, emotion, and meta-cognitive control[J]. Mind & Society (S1593-7879), 2012, 11(1): 107-119.
[55] Sun R.Memory Systems Within a Cognitive Architecture[J]. New Ideas in Psychology (S0732-118X), 2012, 30(2): 227-240.
[56] Brehmer B.The Dynamic OODA Loop: Amalgamating Boyd’s OODA Loop and the Cybernetic Approach to Command and Control[C]. 10th International Command and Control Research Technology Symposium. USA: Command and Control Research Program, 2005: 365-368.
[57] Endsley M R.Toward a Theory of Situation Awareness in Dynamic Systems[J]. Human Factors (S0018-7208), 1995, 37(1): 32-64.
[58] Hu Y, Harabor D, Qin L, et al.Improving the Combination of JPS and Geometric Containers[C]// The International Conference on Automated Planning and Scheduling (ICAPS-2019). CA, USA: AAAI Press, 2019.
[59] Zeng J, Qin L, Hu Y, et al.Integrating a Path Planner and an Adaptive Motion Controller for Navigation in Dynamic Environments[J]. Applied Sciences (S2076-3417), 2019, 9(7): 1384.
[60] Zeng J, Qin L, Hu Y, et al.Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder[J]. Applied Sciences (S2076-3417), 2019, 9(2): 323.
[61] Qin L, Yin Q, Zha Y, et al.Dynamic Detection of Topological Information from Grid-based Generalized Voronoi Diagrams[J]. Mathematical Problems in Engineering (S1024-123X), 2013(10): 1-11.
[62] Yin Q, Qin L, Liu X, et al.Incremental Construction of Generalized Voronoi Diagrams on Pointerless Quadtrees[J]. Mathematical Problems in Engineering (S1024-123X), 2014(1): 1-14.
[63] Yin Q, Qin L, Peng Y, et al.Learning Real-time Search on C-space GVDs[J]. Frontiers of Computer Science (S2095-2228), 2017, 11(6): 1036-1049.
[64] Hu C, Yin Q, Hu Y, et al.Speeding up FastMap for Pathfinding on Grid Maps[C]. ICMA. Tianjin: IEEE, 2019: 2501-2506.
[65] Wang D, Qian X, Quek C, et al.An Interpretable Neural Fuzzy Inference System for Predictions of Underpricing in Initial Public Offerings[J]. Neurocomputing (S0925-2312), 2018, 319: 102-117.
[66] Sun L, Jiao P, Xu K, et al.Modified Adversarial Hierarchical Task Network Planning in Real-time Strategy Games[J]. Applied Sciences (S2076-3417), 2017, 7(9): 872.
[67] Yang W, Peng Y, Yin Q, et al.History-based Single Belief State Generation for Partially Observable Real-time Strategy Games[C]. 2018 Winter Simulation Conference (WSC). Gothenburg, Sweden: IEEE, 2018: 763-773.
[68] Churchill D, Saffidine A, Buro M.Fast Heuristic Search for RTS Game Combat Scenarios[C]. Eighth Artificial Intelligence and Interactive Digital Entertainment Conference. Toronto, CAN: AAAI Press, 2012.
[69] Churchill D, Buro M.Portfolio Greedy Search and Simulation for Large-scale Combat in StarCraft[C]. 2013 IEEE Conference on Computational Intelligence in Games (CIG). Niagara Falls, CAN: IEEE, 2013: 1-8.
[70] Ontañón S, Mishra K, Sugandh N, et al.Case-based Planning and Execution for Real-time Strategy Games[C]. International Conference on Case-Based Reasoning. Belfast, UK: Springer, 2007: 164-178.