Abstract: The depth maps captured by a small depth camera on mobile devices suffer from the problem of severe holes. The Guided Generative Adversarial Network (Guided GAN) based on deep learning is proposed to restore human depth maps with above problems. The high-precision human segmentation features and depth class features are extracted from the monocular RGB image by the guider based on the stacked hourglass network. The holes in the human depth maps are filled by the special generator under the guidance of the extracted human features. In order to get the more realistic results, the discriminator is introduced to optimize the generator. The experimental results show that the proposed method can restore the human depth maps effectively in the existing human datasets and the dataset collected by the small depth camera. It achieves better results than the existing method.

Key words: RGBD camera, human depth data restoration, deep learning, two-stage stacked hourglass network, guided GAN