Spatiotemporal Saliency Detection of Infrared Videos Based on Gestalt-guided Optimization

doi:10.16182/j.issn1004731x.joss.18-0668

Abstract

Abstract: A spatiotemporal saliency detection method based on Gestalt optimization is proposed. A method based on the multi-scale local sparse representation and local contrast measure is proposed to compute the spatial saliency in the infrared videos. A multi-frame symmetric difference approach is adopted to detect the temporal saliency. To get the initial spatiotemporal saliency map, a scheme based on the mutual-consistency is designed to fuse the spatial and temporal saliency maps adaptively. A Gestalt-guided optimization method is designed to calculate the final spatiotemporal saliency map. Experimental results show that the proposed method can effectively detect the spatiotemporal saliency of infrared videos.

Key words: infrared, saliency detection, sparse representation, Gestalt theory

CLC Number:

TP391.9

Wang Xin, Zhang Chunyan, Ning Chen. Spatiotemporal Saliency Detection of Infrared Videos Based on Gestalt-guided Optimization[J]. Journal of System Simulation, 2020, 32(6): 1021-1031.

References

[1] 王仕民, 叶继华, 王明文, 等. 基于多尺度区域协方差的显著性特征提取方法[J]. 系统仿真学报, 2018, 30(7): 2767-2775.
Wang Shimin, Ye Jihua, Wang Mingwen, et al. Saliency Feature Extraction Method Based on Multi-scale Region Covariance [J]. Journal of System Simulation, 2018, 30(7): 2767-2775.
[2] Wang X, Lv G, Xu L. Infrared dim target detection based on visual attention[J]. Infrared Physics & Technology (S1350-4495), 2012, 55(6): 513-521.
[3] 李晓阳, 万丽莉, 李赫男, 等. 结合显著性检测和图割的RGBD 图像共分割算法[J]. 系统仿真学报, 2018, 30(7): 2558-2567.
Li Xiaoyang, Wan Lili, Li Henan, et al. RGBD Image Co-segmentation via Saliency Detection and Graph Cut[J]. Journal of System Simulation, 2018, 30(7): 2558-2567.
[4] Itti L. Automatic foveation for video compression using a neurobiological model of visual attention[J]. IEEE Transactions on Image Processing (S1057-7149), 2004, 13(10): 1304-1318.
[5] Itti L, Koch C, Niebur E. A Model of Saliency-Based Visual Attention for Rapid Scene Analysis[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence (S0162-8828), 1998, 20(11): 1254-1259.
[6] Harel J, Koch C, Perona P. Graph-Based Visual Saliency[C]// Proceedings of International Conference on Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2006: 545-552.
[7] Zhai Y, Shah M. Visual attention detection in video sequences using spatiotemporal cues[C]// Proceedings of ACM International Conference on Multimedia. Santa Barbara, CA, USA: ACM, 2006, 177(1): 815-824.
[8] Hou X, Zhang L. Dynamic Visual Attention: Searching for coding length increments[C]// Proceedings of International Conference on Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2009: 681-688.
[9] Kim H, Kim Y, Sim J Y, et al. Spatiotemporal Saliency Detection for Video Sequences Based on Random Walk With Restart[J]. IEEE Transactions on Image Processing (S1057-7149), 2015, 24(8): 2552-2564.
[10] Wang X, Zhang C Y, Ning C, et al. Detecting Saliency in Infrared Images via Multiscale Local Sparse Representation and Local Contrast Measure[J]. Mathematical Problems in Engineering (S1024-123X), 2017, 5: 2483169.
[11] Dickinson P, Hunter A, Appiah K. A spatially distributed model for foreground segmentation[J]. Image & Vision Computing (S0262-8856), 2009, 27(9): 1326-1335.
[12] Mahapatra D, Gilani S O, Saini M K. Coherency Based Spatio-Temporal Saliency Detection for Video Object Segmentation[J]. IEEE Journal of Selected Topics in Signal Processing (S1932-4553), 2014, 8(3): 454-462.
[13] Yang G, Wang Z, Shitang M U, et al. An Improved Optical Flow Algorithm[J]. Computer Engineering (S1000-3428), 2006, 32(15): 187-188.
[14] Wang Z, Liao K, Xiong J, et al. Moving object detection based on temporal information[J]. IEEE Signal Processing Letters (S1070-9908), 2014, 21(11): 1403-1407.
[15] Wang X, Zhang Y, Ning C. A novel visual saliency detection method for infrared video sequences[J]. Infrared Physics & Technology (S1350-4495), 2017, 87: 91-103.
[16] Ren Z, Gao S, Chia L T, et al. Regularized feature reconstruction for spatio-temporal saliency detection[J]. IEEE Transactions on Image Processing (S1057-7149), 2013, 22(8): 3120-3132.
[17] Kim W, Kim C. Spatiotemporal Saliency Detection Using Textural Contrast and Its Applications[J]. IEEE Transactions on Circuits & Systems for Video Technology (S1051-8215), 2014, 24(4): 646-659.
[18] Li Y, Tan Y, Yu J G, et al. Kernel regression in mixed feature spaces for spatio-temporal saliency detection[J]. Computer Vision & Image Understanding (S1077-3142), 2015, 135: 126-140.
[19] Wang X, Ning C, Xu L. Saliency detection using mutual consistency-guided spatial cues combination[J]. Infrared Physics & Technology (S1350-4495), 2015, 72: 106-116.
[20] Ren Z, Gao S, Chia L T, et al. Regularized feature reconstruction for spatio-temporal saliency detection[J]. IEEE Transactions on Image Processing (S1057-7149), 2013, 22(8): 3120-3132.
[21] King D B, Wertheimer M. Max Wertheimer and Gestalt Theory[M]. Piscataway: Transaction Publishers, 2005: 438.
[22] 李炜烨. 基于格式塔理论的建筑物区域提取方法研究[D]. 南昌: 南昌航空大学, 2014.
Li W Y. Building area detection method based on the principle of gestalt theory [D]. Nanchang: Nanchang Hangkong University, 2014.
[23] Davis J W. OTCBVS Benchmark Dataset Collection[EB/OL]. (2005-06-20)[2018-01-03]. http://www. cse.ohio-state.edu/otcbvs- bench/.
[24] Wang X, Ning C, Xu L. Spatiotemporal saliency model for small moving object detection in infrared videos[J]. Infrared Physics & Technology (S1350-4495), 2015, 69: 111-117.