Action Recognition by Improved Dense Trajectories

doi:10.16182/j.issn1004731x.joss.201709023

Abstract

Abstract: In recent years, with strong robustness to fast irregular motion, the method of human motion representation based on dense trajectories has been used more and more in the field of behavior recognition. However, the relative motion of the background caused by the motion of the camera has a great influence on the extraction of the trajectories. In order to estimate the camera motion, the speed up robust feature (SURF) descriptor was used to match the feature points of each frame. Since the human motion and the camera motion were not same, human detection was added to remove inconsistent matches. Finally, multi instance learning (MIL) was used to classify and recognize the behavior. Experiment results demonstrate the effectiveness of the approach on the UT-interaction dataset.

Key words: dense trajectories, the estimate of camera motion, human detection, multi instance learning

CLC Number:

TP391

Xu Peizhen, Yu Zhibin, Jin Weidong, Jiang Haiying. Action Recognition by Improved Dense Trajectories[J]. Journal of System Simulation, 2017, 29(9): 2053-2058.

References

[1] Zhu J, Zhao Y, Tang J.Automatic Recognition of Radar Signals Based on Time-frequency Image Character[J]. Defence Science Journal (S0011-748X), 2013, 63(3): 1-6.
[2] Wang H, Klaser A, Schmid C, et al.Action Recognition by Dense Trajectories[C]// Proceeding of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. USA: IEEE Computer Society, 2011: 3169-3176.
[3] Aggarwal J K.Recognition of Human Activities[M]// Combinatorial Image Analysis. Berlin, Heidelberg, Germany: Springer, 2011: 1-4.
[4] Laptev I, Lindeberg T.On Space-Time Interest Points[J]. International Journal of Computer Vision (S0920-5691), 2005, 64(2/3): 107-123.
[5] Dollar P, Rabaud V, Cottrell G, et al.Behavior Recognition Via Sparse Spatio-temporal Features[C]// 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance. USA: IEEE Computer Society, 2005: 65-72.
[6] Messing R, Pal C, Kautz H.Activity Recognition Using the Velocity Histories of Tracked Keypoints[C]// Proceedings IEEE International Conference on Computer Vision. USA: IEEE, 2009: 104-111.
[7] Matikainen P, Hebert M, Sukthankar. Trajectons: Action Recognition Through the Motion Analysis of Tracked Features[C]// Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on. USA: IEEE, 2009: 514-521.
[8] Wang H, Klaser A, Schmid C, et al.Dense Trajectories and Motion Boundary Descriptors for Action Recognition[J]. International Journal of Computer Vision (S0920-5691), 2013, 103(1): 60-78.
[9] Szeliski R.Image Alignment and Stitching: a Tutorial[J]. Foundations & Trends in Computer Graphics & Vision (S1572-2740), 2004, 2(1): 101-104.
[10] Bay H, Tuytelaars T, Gool L V.SURF: Speeded Up Robust Features[J]. Computer Vision & Image Understanding (S1077-3142), 2006, 110(3): 404-417.
[11] Gaidon A, Harchaoui Z, Schmid C.Recognizing Activities with Cluster-trees of Track lets [C]// British Machine Vision Conference. United Kingdom: BNVA Press, 2012: 1-6.
[12] Farneback G.Two-frame motion estimation based on polynomial expansion[C]// Scandinavian Conference on Image Analysis. Germany: Springer-Verlag, 2003: 363-370.
[13] Shi J, Tomasi C.Good Features to Track[M]. USA: Cornell University, 1993.
[14] Fischler M A, Bolles R C.Rand on Sample Consensus: a Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography[J]. Communications of the ACM (S0001-0782), 1981, 24(6): 381-395.
[15] Wang H, Kläser A, Schmid C, et al.Dense Trajectories and Motion Boundary Descriptors for Action Recognition[J]. International Journal of Computer Vision (S0920-5691), 2013, 103(1): 60-79.
[16] Prest A, Schmid C, Ferrari V.Weakly supervised learning of interactions between humans and objects[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence (S0162-8828), 2012, 34(3): 601-614.
[17] Watanabe T, Ito S, Yokoi K.Co-occurrence Histograms of Oriented Gradients for Pedestrian Detection[M]// Advances in Image and Video Technology. Berlin Heidelberg, Germany: Springer, 2009: 37-47.
[18] Peng X, Qiao Y, Peng Q, et al.Exploring Motion Boundary based Sampling and Spatial-Temporal Context Descriptors for Action Recognition[C]// British Machine Vision Conference. United Kingdom: BMVC, 2013: 1-11.
[19] Farneback G.Two-frame motion estimation based on polynomial expansion[C]// Scandinavian Conference on Image Analysis. Germany: Springer-Verlag, 2003: 363-370.
[20] Wang H, Klaser A, Schmid C, et al.Dense Trajectories and Motion Boundary Descriptors for Action Recognition[J]. International Journal of Computer Vision (S0920-5691), 2013, 103(1): 60-79.
[21] Zhou Z H, Zhang M L.Neural Networks for Multi Instance Learning [R]// International Conference on Intelligent Information Technology. China: Nanjing University, 2002: 1-14.
[22] Ryoo M S, Aggarwal J K.Spatio-temporal Relationship Match: Video Structure Comparison for Recognition of Complex Human Activities[C]// IEEE, International Conference on Computer Vision. USA: IEEE Xplore, 2009: 1593-1600.
[23] Alonso P P, Marcin M, Ian R, et al.Structured Learning of Human Interactions in TV Shows[J]. IEEE Transactions on Software Engineering (S0098-5589), 2012, 34(12): 2441-2453.
[24] Yang L, Gao C, Meng D, et al.A Novel Groups parsity-Optimization-Based Feature Selection Model for Complex Interaction Recognition [M]// Computer Vision-ACCV2014. Germany: Springer International Publishing, 2015: 508-521.