A deep spatio-temporal RNNs based coordinated operational action recognition

doi:10.16182/j.issn1004731x.joss.201803004

Abstract

Abstract: To address the issues of large feature space, numerous model parameters and slow training speed in coordinated operation action recognition, a coordinated operational action recognition method based on a deep spatio-temporal recurrent neural network is proposed. In this method, a warped region generation mechanism is introduced to divide the whole battlefield into sub-battlefield. Meanwhile, a hierarchical recurrent neural network is constructed using spatio-temporal graph model, which is applied to the generated sub-battlefield to recognize coordinated operational action. Additionally, the recognized coordinated operational actions of sub-battlefields are merged to find out all coordinated operational actions based on the principle of transitivity of coordinated operational actions in local battlefield. Experiment results suggest that the proposed method possesses higher accuracy.

Key words: coordinated operation, hierarchical recurrent neural network, coordinated operation action recognition, spatio-temporal graph, situation comprehension

CLC Number:

TP183
E919

Yi Zhuo, Liao Ying, Hu Xiaofeng, Du Xuehui, Zhu Feng. A deep spatio-temporal RNNs based coordinated operational action recognition[J]. Journal of System Simulation, 2018, 30(3): 793-800.

References 21

[1]	宋元, 王永春. 海上防空作战态势估计理论及应用[M]. 北京: 国防工业出版社, 2014. Song Yuan, Wang Yongchun. The theory and application of sea air defense combat situation estimation[M]. Beijing: National Defense Industry Press, 2014.
[2]	Peng P, Wen Y, Yang Y, et al.Multiagent Bidirectionally- Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games[J]. 2017.
[3]	Blunsden S, Fisher R, Andrade E.Recognition of coordinated multi agent activities, the individual vs the group[J]. Language Culture & Curriculum (S0790-8318), 2006, 22(2): 157-174.
[4]	Min F, Groen F C A, Hao L. Dynamic Partition of Collaborative Multiagent Based on Coordination Trees[C]. Intelligent autonomous systems 12: proceedings of the 12th International Conference IAS-12, held June 26-29, 2012 Jeju Island, Korea. Springer, 2013, 2: 503-510.
[5]	Fang M, Groen F C A, Li H, et al. Collaborative multi-agent reinforcement learning based on a novel coordination tree frame with dynamic partition[J]. Engineering Applications of Artificial Intelligence (S0952-1976), 2014, 27(1): 191-198.
[6]	Liu L, Wang S, Su G, et al.Towards complex activity recognition using a Bayesian network-based probabilistic generative framework[J]. Pattern Recognition (S0031-3203), 2017, 68(C): 295-309.
[7]	Swears E, Hoogs A, Ji Q, et al.Complex Activity Recognition Using Granger Constrained DBN (GCDBN) in Sports and Surveillance Video[C]. IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 2014: 788-795.
[8]	Diks C, Panchenko V.A new statistic and practical guidelines for nonparametric Granger causality testing[J]. Journal of Economic Dynamics & Control (S0165-1889), 2006, 30(9): 1647-1669.
[9]	Liu L, Yang Y, Govindarajan L N, et al. An Interval-Based Bayesian Generative Model for Human Complex Activity Recognition[J]. arXiv:1701.00903, 2017.
[10]	Zhu W, Lan C, Xing J, et al.Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks[C]. Thirtieth AAAI Conference on Artificial Intelligence. AAAI Press, 2016: 3697-3703.
[11]	Taylor G W, Fergus R, Lecun Y, et al.Convolutional learning of spatio-temporal features[C]. European Conference on Computer Vision. Springer-Verlag, 2010: 140-153.
[12]	Le Q V, Zou W Y, Yeung S Y, et al.Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis[C]. Computer Vision and Pattern Recognition. IEEE, 2011: 3361-3368.
[13]	Ji S, Xu W, Yang M, et al.3D Convolutional Neural Networks for Human Action Recognition[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence (S0162-8828), 2013, 35(1): 221.
[14]	Wang Keze, Wang Xiaolong, Lin Liang, et al.3D Human Activity Recognition with Reconfigurable Convolutional Neural Networks[C]. Proceedings of the 2014 ACM Conference on Multimedia, 2014: 97-106.
[15]	Chen B.Deep Learning of Invariant Spatio-Temporal Features from Video[J]. University of British Columbia, 2010.
[16]	Jain A, Zamir A R, Savarese S, et al.Structural-RNN: Deep Learning on Spatio-Temporal Graphs[C]. Computer Vision and Pattern Recognition. IEEE, 2016: 5308-5317.
[17]	Von L T, Brodkorb F, Roskosch P, et al.MobilityGraphs: Visual Analysis of Mass Mobility Dynamics via Spatio-Temporal Graphs and Clustering[J]. IEEE Transactions on Visualization & Computer Graphics (S1077-2626), 2016, 22(1): 11-20.
[18]	Mondo G D, Rodríguez M A, Claramunt C, et al.Modeling consistency of spatio-temporal graphs[J]. Data & Knowledge Engineering (S0169-023X), 2013, 84(3): 59-80.
[19]	Yu B, Yin H, Zhu Z. Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting[J]. arXiv:1709.04875, 2017.
[20]	Uijlings J R, Sande K E, Gevers T, et al.Selective Search for Object Recognition[J]. International Journal of Computer Vision (S0920-5691), 2013, 104(2): 154-171.
[21]	欧微, 柳少军, 贺筱媛, 等. 基于时序特征编码的目标战术意图识别算法[J]. 指挥控制与仿真, 2016, 38(6): 36-41. OU Wei, Liu Shaojun, He Xiaoyuan, et al. Tactical intention recognition algorithm based on encoded temporal features[J]. Command Control & Simulation, 2016, 38(6): 36-41.