基于深度神经网络的多视角人体动作识别

doi:10.16182/j.issn1004731x.joss.19-0448

系统仿真学报 ›› 2021, Vol. 33 ›› Issue (5): 1019-1030.doi: 10.16182/j.issn1004731x.joss.19-0448

基于深度神经网络的多视角人体动作识别

赵瑛^1,2, 陆耀¹, 张健³, 梁启弟³, 龙炜¹

1.北京理工大学智能信息技术北京市重点实验室,北京 100081;
2.北京联合大学师范学院,北京 100011;
3.中南大学计算机学院,长沙 410083

收稿日期:2019-08-26 修回日期:2019-10-08 出版日期:2021-05-18 发布日期:2021-06-09
通讯作者: 陆耀(1958-),男,博士,教授,研究方向为神经网络、图像和信号处理、模式识别。E-mail：vis_yl@bit.edu.cn
作者简介:赵瑛(1977-),女,博士,副教授,研究方向为人体行为分析、机器学习、智慧教育。E-mail：sftzhaoying@buu.edu.cn
基金资助:
国家自然科学基金(61273273); 国家重点研发计划(2017YFC0112001)

Multi-view Human Action Recognition Based on Deep Neural Network

Zhao Ying^1,2, Lu Yao¹, Zhang Jian³, Liang Qidi³, Long Wei¹

1. Beijing Laboratory of Intelligent Information Technology, Beijing Institute of Technology, Beijing 100081, China;
2. Teachers College, Beijing Union University, Beijing 100011, China;
3. School of Computer Science and Engineering, Central South University, Changsha 410083, China

Received:2019-08-26 Revised:2019-10-08 Online:2021-05-18 Published:2021-06-09

摘要/Abstract

摘要： 为提高多视角人体动作识别的精度,提出了一种新的深度神经网络模型——CNN+CA(Convolutional Neural Network plus Context Attention)模型和一种基于序列匹配的识别方法。利用卷积神经网络自动学习出多视角融合特征;引入上下文注意力模块自动突出特征中有利于识别的区域,进一步提高特征的判别力;通过基于序列匹配的方法实现人体动作识别。在IXMAS 数据集和i3DPost 数据集上的实验结果表明,所提方法在识别精度上超过了其他同类方法。

关键词: 多视角, 人体动作识别, 卷积神经网络, 上下文注意力, 序列匹配

Abstract: A novel deep neural network named CNN+CA(Convolutional Neural Network plus Context Attention) model is constructed and a new recognition algorithm based on sequence matching is presented to improve the recognition accuracy of MVHAR (Multi-view Human Action Recognition). A CNN(Convolutional Neural Network) is designed to automatically learn multi-view fusion features; the CA (Context Attention) module is introduced to selectively focus on the parts of the features that are relevant for the recognition task; the proposed recognition algorithm based on sequence matching is used to realize MVHAR. The experimental results on the IXMAS dataset and the i3DPost dataset demonstrate that the recognition accuracy of the proposed method is higher than those of the state-of-the-art MVHAR methods.

Key words: multi-view, human action recognition, convolutional neural network, context attention, sequence matching

中图分类号:

TP391

赵瑛, 陆耀, 张健, 梁启弟, 龙炜. 基于深度神经网络的多视角人体动作识别[J]. 系统仿真学报, 2021, 33(5): 1019-1030.

Zhao Ying, Lu Yao, Zhang Jian, Liang Qidi, Long Wei. Multi-view Human Action Recognition Based on Deep Neural Network[J]. Journal of System Simulation, 2021, 33(5): 1019-1030.

参考文献

[1] Sharma S, Kiros R, Salakhutdinov R.Action Recognition Using Visual Attention[C]// International Conference on Learning Representations Workshop. San Juan, Puerto Rico, USA: Springer, 2016: 1-11.
[2] Jaderberg M, Simonyan K, Zisserman A, et al.Spatial Transformer Networks[C]// Advances in Neural Information Processing Systems. Montréal CA: Springer, 2015: 2017-2025.
[3] Aryanfar A, Yaakob R, Halin A A, et al.Multi-view Human Action Recognition Using Wavelet Data Reduction and Multi-Class Classification[J]. Procedia Computer Science (S1877-0509), 2015, 100(62): 585-592.
[4] Liu C, Li Z, Shi X, et al.Learning a Mid-level Representation for Multi-View Action Recognition[J]. Advances in Multimedia (S1687-5680), 2018: 1-10.
[5] Zhen X, Shao L, Maybank S, et al.Handcrafted vs. Learned Representations for Human Action Recognition[J]. Image and Vision Computing (S0262-8856), 2016, 55(2): 39-41.
[6] Putra P U, Shima K, Shimatani K.Markerless Human Activity Recognition Method Based on Deep Neural Network Model Using Multiple Cameras[C]// IEEE International Conference on Control, Decision and Information Technologies. Thessaloniki, Greece: IEEE, 2018: 13-18.
[7] Chuanxu W, Guofeng H, Yun L.Multi-views Action Recognition on Deep Learning and K-SVD[J]. Journal of Physics Conference Series (S1742-6588), 2019, 1176(6): 062015.
[8] Kavi R, Kulathumani V, Rohit F, et al.Multiview Fusion for Activity Recognition Using Deep Neural Networks[J]. Journal of Electronic Imaging (S1017-9909), 2016, 25(4): 043010.
[9] Weinland D, Ronfard R, Boyer E.Free Viewpoint Action Recognition Using Motion History Volumes[J]. Computer Vision and Image Understanding (S1077-3142), 2006, 104(2/3): 249-257.
[10] Gkalelis N, Kim H, Hilton A, et al.The i3DPost Multi-View and 3D Human Action/Interaction Database[C]// IEEE Conference for Visual Media Production. London, UK: IEEE, 2009: 159-168.
[11] Sargano A, Angelov P, Habib Z.Human Action Recognition From Multiple Views Based on View-invariant Feature Descriptor Using Support Vector Machines[J]. Applied Sciences (S2076-3417), 2016, 6(10): 309-322.
[12] Zhang J, Zhang L, Shum H P H, et al. Arbitrary View Action Recognition Via Transfer Dictionary Learning on Synthetic Training Data[C]// IEEE International Conference on Robotics and Automation. Stockholm, Sweden: IEEE, 2016: 1678-1684.
[13] Liu C W, Pei M T, Wu X X, et al.Learning a Discriminative Mid-Level Feature for Action Recognition[J]. Science China-Information Sciences (S1862-2836), 2014, 57(5): 1-13.
[14] Zhang J, Shum H P, Han J, et al.Action Recognition From Arbitrary Views Using Transferable Dictionary Learning[J]. IEEE Transactions on Image Processing (S1057-7149), 2018, 27(10): 4709-4723.
[15] Chalearnnetkul P, Suvonvorn N.Multiview Layer Fusion Model for Action Recognition Using RGBD Images[J]. Computational Intelligence and Neuroscience (S1687-5265), 2018: 1-22.
[16] Iosifidis A, Tefas A, Pitas I.Regularized Extreme Learning Machine for Multi-view Semi-supervised Action Recognition[J]. Neurocomputing (S0925-2312), 2014, 145(5): 250-262.
[17] Sadek S, Al-Hamadi A, Krell G, et al.Affine-Invariant Feature Extraction for Activity Recognition[J]. ISRN Machine Vision (S2090-7796), 2013(1): 1-7.
[18] Iosifidis A, Tefas A, Nikolaidis N, et al.Multi-view Human Movement Recognition Based on fuzzy Distances and Linear Discriminant Analysis[J]. Computer Vision and Image Understanding (S1077-3142), 2012, 116(3): 347-360.
[19] Holte M B, Moeslund T B, Nikolaidis N, et al.3D Human Action Recognition for Multi-view Camera Systems[C]// International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission. Hangzhou, China: IEEE, 2011: 342-349.

基于深度神经网络的多视角人体动作识别

Multi-view Human Action Recognition Based on Deep Neural Network

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张立峰, 王会忍. 基于卷积神经网络及有限元仿真的电容层析成像图像重建[J]. 系统仿真学报, 2022, 34(4): 712-718.
[2]	敖邦乾, 杨莎, 令狐金卿, 叶振环. 基于级联神经网络疲劳驾驶检测系统设计[J]. 系统仿真学报, 2022, 34(2): 323-333.
[3]	仝卫国, 庞雪纯, 朱赓宏. 基于卷积神经网络的气液两相流流型识别方法[J]. 系统仿真学报, 2021, 33(4): 883-891.
[4]	马榕, 陈秋瑞, 张晗, 梅铮, 王锐, 魏伟. 基于单目深度估计的低功耗视觉里程计[J]. 系统仿真学报, 2021, 33(12): 3001-3011.
[5]	黄欣, 方钰, 顾梦丹. 基于卷积神经网络的X线胸片疾病分类研究[J]. 系统仿真学报, 2020, 32(6): 1188-1194.
[6]	樊名鲁, 王艳, 纪志成. 基于特征生成的轴承不均衡数据故障诊断[J]. 系统仿真学报, 2020, 32(12): 2438-2448.
[7]	王红军, 黎邹邹, 邹湘军. 基于Adaboost与CNN的木材表面缺陷检测[J]. 系统仿真学报, 2019, 31(8): 1636-1645.
[8]	殷小静, 胡晓峰, 荣明, 刘海洋, 杨永利. 体系贡献率评估方法研究综述与展望[J]. 系统仿真学报, 2019, 31(6): 1027-1038.
[9]	卢裕秋, 孙金玉, 马世伟. 基于深度卷积神经网络的运动目标检测方法[J]. 系统仿真学报, 2019, 31(11): 2275-2280.
[10]	石祥滨, 钟健, 刘翠微, 刘芳, 张德园. 小样本高光谱遥感图像深度学习方法[J]. 系统仿真学报, 2018, 30(7): 2744-2752.
[11]	夏旻, 申茂阳, 王舰锋, 王阳光. 基于卷积神经网络的卫星云图云量计算[J]. 系统仿真学报, 2018, 30(5): 1623-1630.
[12]	廖鹰, 易卓, 胡晓峰, 田园, 陶九阳. 基于三维卷积神经网络的战场聚集行为预测[J]. 系统仿真学报, 2018, 30(3): 801-808.
[13]	吴昀璞, 金炜东, 黄颖坤. 基于多域融合CNN的高速列车转向架故障检测[J]. 系统仿真学报, 2018, 30(11): 4492-4497.
[14]	高尚兵, 颉正, 潘志庚, 覃方哲, 李锐. 一种新的路面裂缝自动检测算法[J]. 系统仿真学报, 2017, 29(9): 2009-2015.
[15]	王松, 刘复昌, 黄骥, 许威威, 董洪伟. 基于卷积神经网络的深度图姿态估计算法研究[J]. 系统仿真学报, 2017, 29(11): 2618-2623.