Abstract: In this paper, an augmented reality scheme that can achieve the virtual-real objects' interaction effectively is proposed. The depth camera, such as Kinect, is adopted to capture the depth image of scene, and then the segmentation and dynamic tracking of the scene objects are implemented based on the depth image. In the scene segmentation, we propose a scene segmentation strategy based on the prior knowledge. The large plane objects existing in the scene are firstly recognized according to the prior knowledge. Then, the clustering segmentation is applied to the remaining points to get the scene objects. This strategy is very efficient for the indoor desktop-scene segmentation. In the object tracking, we adopt the three dimensional tracking algorithm based on particle filtering. The tracking process can also be accelerated by using the prior large plane information. This paper provides an efficient scheme for implementing the virtual-real objects' interaction.

Key words: augmented reality, depth image segmentation, object tracking, virtual-real objects interaction