关键词: 动态视觉传感器, 手势识别, 多维投影, 时空事件帧, 脉冲神经网络

Abstract:

Vision-based gesture recognition is a commonly used means of human-computer interaction in the fields of virtual reality and game simulation. In practical applications, rapid changes in gesture movements will lead to blurred imaging with traditional RGB cameras or depth cameras, which brings great challenges to gesture recognition. To solve the above problems, a dynamic visual data gesture recognition method based on a multi-dimensional projection spatiotemporal event frame (STEF) is proposed by a using dynamic vision sensor to capture high-speed gesture movement information. The spatiotemporal information is embedded in the data projection surface and fused to form a multi-dimensional projection STEF, which overcomes the limitation of the time-domain information loss of the existing event frame expression method of dynamic visual information and improves the feature expression ability of dynamic visual sensing data. On this basis, advanced spiking neural networks are used to classify STEFs to realize gesture recognition. The recognition accuracy of the above method on the public dataset reaches 96.67%, which is better than similar methods, indicating that the proposed method can significantly improve the accuracy of gesture recognition in dynamic visual sensing data.

Key words: dynamic vision sensor, gesture recognition, multi-dimensional projection, spatiotemporal event frame, spiking neural network