Multi-view Human Action Recognition Based on Deep Neural Network

doi:10.16182/j.issn1004731x.joss.19-0448

Abstract

Abstract: A novel deep neural network named CNN+CA(Convolutional Neural Network plus Context Attention) model is constructed and a new recognition algorithm based on sequence matching is presented to improve the recognition accuracy of MVHAR (Multi-view Human Action Recognition). A CNN(Convolutional Neural Network) is designed to automatically learn multi-view fusion features; the CA (Context Attention) module is introduced to selectively focus on the parts of the features that are relevant for the recognition task; the proposed recognition algorithm based on sequence matching is used to realize MVHAR. The experimental results on the IXMAS dataset and the i3DPost dataset demonstrate that the recognition accuracy of the proposed method is higher than those of the state-of-the-art MVHAR methods.

Key words: multi-view, human action recognition, convolutional neural network, context attention, sequence matching

CLC Number:

TP391

Zhao Ying, Lu Yao, Zhang Jian, Liang Qidi, Long Wei. Multi-view Human Action Recognition Based on Deep Neural Network[J]. Journal of System Simulation, 2021, 33(5): 1019-1030.

References

[1] Sharma S, Kiros R, Salakhutdinov R.Action Recognition Using Visual Attention[C]// International Conference on Learning Representations Workshop. San Juan, Puerto Rico, USA: Springer, 2016: 1-11.
[2] Jaderberg M, Simonyan K, Zisserman A, et al.Spatial Transformer Networks[C]// Advances in Neural Information Processing Systems. Montréal CA: Springer, 2015: 2017-2025.
[3] Aryanfar A, Yaakob R, Halin A A, et al.Multi-view Human Action Recognition Using Wavelet Data Reduction and Multi-Class Classification[J]. Procedia Computer Science (S1877-0509), 2015, 100(62): 585-592.
[4] Liu C, Li Z, Shi X, et al.Learning a Mid-level Representation for Multi-View Action Recognition[J]. Advances in Multimedia (S1687-5680), 2018: 1-10.
[5] Zhen X, Shao L, Maybank S, et al.Handcrafted vs. Learned Representations for Human Action Recognition[J]. Image and Vision Computing (S0262-8856), 2016, 55(2): 39-41.
[6] Putra P U, Shima K, Shimatani K.Markerless Human Activity Recognition Method Based on Deep Neural Network Model Using Multiple Cameras[C]// IEEE International Conference on Control, Decision and Information Technologies. Thessaloniki, Greece: IEEE, 2018: 13-18.
[7] Chuanxu W, Guofeng H, Yun L.Multi-views Action Recognition on Deep Learning and K-SVD[J]. Journal of Physics Conference Series (S1742-6588), 2019, 1176(6): 062015.
[8] Kavi R, Kulathumani V, Rohit F, et al.Multiview Fusion for Activity Recognition Using Deep Neural Networks[J]. Journal of Electronic Imaging (S1017-9909), 2016, 25(4): 043010.
[9] Weinland D, Ronfard R, Boyer E.Free Viewpoint Action Recognition Using Motion History Volumes[J]. Computer Vision and Image Understanding (S1077-3142), 2006, 104(2/3): 249-257.
[10] Gkalelis N, Kim H, Hilton A, et al.The i3DPost Multi-View and 3D Human Action/Interaction Database[C]// IEEE Conference for Visual Media Production. London, UK: IEEE, 2009: 159-168.
[11] Sargano A, Angelov P, Habib Z.Human Action Recognition From Multiple Views Based on View-invariant Feature Descriptor Using Support Vector Machines[J]. Applied Sciences (S2076-3417), 2016, 6(10): 309-322.
[12] Zhang J, Zhang L, Shum H P H, et al. Arbitrary View Action Recognition Via Transfer Dictionary Learning on Synthetic Training Data[C]// IEEE International Conference on Robotics and Automation. Stockholm, Sweden: IEEE, 2016: 1678-1684.
[13] Liu C W, Pei M T, Wu X X, et al.Learning a Discriminative Mid-Level Feature for Action Recognition[J]. Science China-Information Sciences (S1862-2836), 2014, 57(5): 1-13.
[14] Zhang J, Shum H P, Han J, et al.Action Recognition From Arbitrary Views Using Transferable Dictionary Learning[J]. IEEE Transactions on Image Processing (S1057-7149), 2018, 27(10): 4709-4723.
[15] Chalearnnetkul P, Suvonvorn N.Multiview Layer Fusion Model for Action Recognition Using RGBD Images[J]. Computational Intelligence and Neuroscience (S1687-5265), 2018: 1-22.
[16] Iosifidis A, Tefas A, Pitas I.Regularized Extreme Learning Machine for Multi-view Semi-supervised Action Recognition[J]. Neurocomputing (S0925-2312), 2014, 145(5): 250-262.
[17] Sadek S, Al-Hamadi A, Krell G, et al.Affine-Invariant Feature Extraction for Activity Recognition[J]. ISRN Machine Vision (S2090-7796), 2013(1): 1-7.
[18] Iosifidis A, Tefas A, Nikolaidis N, et al.Multi-view Human Movement Recognition Based on fuzzy Distances and Linear Discriminant Analysis[J]. Computer Vision and Image Understanding (S1077-3142), 2012, 116(3): 347-360.
[19] Holte M B, Moeslund T B, Nikolaidis N, et al.3D Human Action Recognition for Multi-view Camera Systems[C]// International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission. Hangzhou, China: IEEE, 2011: 342-349.