基于RGB-D摄像机与IMU数据融合的动作捕捉系统

系统仿真学报 ›› 2015, Vol. 27 ›› Issue (10): 2582-2588.

基于RGB-D摄像机与IMU数据融合的动作捕捉系统

张瑒, 许林, 孙广毅

南开大学机器人与信息自动化研究所天津市智能机器人重点实验室, 天津 300071

收稿日期:2015-06-07 修回日期:2015-07-30 出版日期:2015-10-08 发布日期:2020-08-07
作者简介:张瑒(1991-),男,辽宁,硕士生,研究方向为惯性导航;许林(1975-),男,四川,副教授,研究方向为无线传感网络;孙广毅(1981-),男,天津,副教授,研究方向为虚拟现实与微机电系统。
基金资助:
国家高技术研究发展计划(2013AA041102)

Motion Tracking System by Direct Fusion of RGB-D Camera and Micro-IMU Sensors

Zhang Yang, Xu Lin, Sun Guangyi

Tianjin Key Laboratory of Intelligent Robotics, Institute of Robotics and Automatic Information System, Nankai University, Tianjin 300071, China

Received:2015-06-07 Revised:2015-07-30 Online:2015-10-08 Published:2020-08-07

摘要/Abstract

摘要： 近年来,低功耗全身动作捕捉在计算机视觉与自动化等领域都获得了广泛的关注。单目深度视觉与基于惯性的动作捕捉系统以其相对低廉的价格、良好的捕捉效果成为中小型用户的首选。然而,这两种系统却具有包括身体遮挡与误差漂移等一系列固有缺陷。因此创新性地提出一种基于RGB-D摄像机与惯性测量单元（IMU）数据融合的动作捕捉系统。该系统引入快速遮挡检测、动态阈值判断与权值分配策略,集成惯性测量技术和光学捕捉技术的优点,成本低、不易受环境影响。实验表明基于本方法的动作捕捉系统在精度、可靠性与稳定性上都具有良好的表现。

关键词: 动作捕捉, 数据融合方法, Kinect, 惯性测量单元

Abstract: Recently, low cost full Body Motion Tracking attracted tremendous attentions in the fields of Computer Vision and Automation. Single depth camera-and inertial-based motion tracking systems rapidly have become two primary options for small-business and individual consuming markets, due to their relatively inexpensive price and comparable performance. However, both depth camera-and inertial-based motion tracking systems have inherent disadvantages which cannot be eliminated theoretically and experimentally. For example, depth camera tracking mostly failed in the presence of occlusion while inertial sensor tracking usually suffered from drift errors. Therefore, a novel multiple-sensor fusion approach which seamlessly combined the data measured from a RGB-D Camera and several inertial measurement units (IMU) sensors was presented. The method had the characteristic of low cost and was less likely to be influenced by the environment by introducing several new algorithms, e.g., fast occlusion detection, dynamic threshold determination, and weighted value assignment. Experimental studies show that the method shows superior performance in terms of accuracy, reliability, and robustness.

Key words: motion tracking system, data fusion, Kinect, IMU

中图分类号:

TP391.9

张瑒, 许林, 孙广毅. 基于RGB-D摄像机与IMU数据融合的动作捕捉系统[J]. 系统仿真学报, 2015, 27(10): 2582-2588.

Zhang Yang, Xu Lin, Sun Guangyi. Motion Tracking System by Direct Fusion of RGB-D Camera and Micro-IMU Sensors[J]. Journal of System Simulation, 2015, 27(10): 2582-2588.

参考文献

[1] Baak A, Rosenhahn B, Müller M, et al.Stabilizing motion tracking using retrieved motion priors[C]//Computer Vision, 2009 IEEE 12th International Conference on. USA: IEEE, 2009: 1428-1435.
[2] Bălan Alexandru O, et al. Detailed human shape and pose from images[C]// Computer Vision and Pattern Recognition, 2007. CVPR'07, IEEE Conference on. USA: IEEE, 2007.
[3] Herda Lorna, Raquel Urtasun, Pascal Fua.Hierarchical implicit surface joint limits to constrain video-based motion capture. Computer Vision-ECCV 2004. Germany: Springer Berlin Heidelberg, 2004: 405-418.
[4] Ioffe, Sergey, David Forsyth. Human tracking with mixtures of trees[C]// Computer Vision, 2001, ICCV 2001, Proceedings, Eighth IEEE International Conference on. Vol. 1. USA: IEEE, 2001.
[5] Agarwal Ankur, Bill Triggs.Recovering 3D human pose from monocular images[C]// Pattern Analysis and Machine Intelligence, IEEE Transactions on, 28.1. USA: IEEE, 2006: 44-58.
[6] Elgammal Ahmed, Chan-Su Lee.Inferring 3D body pose from silhouettes using activity manifold learning[C]// Computer Vision and Pattern Recognition, 2004, CVPR 2004, Proceedings of the 2004 IEEE Computer Society Conference on, Vol. 2. USA: IEEE, 2004.
[7] Shakhnarovich Gregory, Paul Viola, Trevor Darrell.Fast pose estimation with parameter-sensitive hashing[C]// Computer Vision, 2003, Proceedings, Ninth IEEE International Conference on. USA: IEEE, 2003.
[8] Machida E, Cao M, Murao T, et al. Human Motion Tracking of Mobile Robot with Kinect 3D Sensor[C]// SICE Annual Conference (SICE), 2012 Proceedings of. IEEE. USA: IEEE, 2012: 2207-2211.
[9] Roetenberg Daniel, Henk Luinge, Per Slycke. Xsens MVN: full 6DOF human motion tracking using miniature inertial sensors [Z]// Xsens Motion Technologies BV, Tech. Rep (2009).
[10] Pons-Moll G, Baak A, Helten T, et al. Multisensor-fusion for 3d full-body human motion capture[C]// Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. USA: IEEE, 2010: 663-670.
[11] Ziegler Jakob, et al. Accurate human motion capture in large areas by combining IMU-and laser-based people tracking[C]// Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on. USA: IEEE, 2011.
[12] 刘博, 安建成. 基于关键姿势的人体动作识别[J]. 电视技术, 2014, (5): 38-41.
[13] 冷晶晶. 基于Kinect骨架信息的人体动作识别[J]. 数字技术与应用, 2014, 9: 80-80.
[14] 郭志虎, 费洁, 钱峰. 基于微惯性全姿态测量的人体运动捕捉系统研究[J]. 微型电脑应用, 2013, (2): 29-32.
[15] Madgwick S O H, Harrison A J L, Vaidyanathan R, Estimation of IMU and MARG orientation using a gradient descent algorithm[C]// IEEE International Conference on Rehabilitation Robotics. USA: IEEE, 2011: 1-7.