Single-frame Image Motion Parallax Key Point Estimation Combined with Self-supervised Learning

doi:10.16182/j.issn1004731x.joss.21-FZ0708

Abstract

Abstract: The motion parallax key point FOE (Focus of Expansion) is an important parameter of railway catenary video inspection. The current method of calculating FOE requires multi-frame image matching estimation, which has high time complexity. Aiming at the single-frame image FOE estimation, a single-frame image FOE estimation algorithm fused with self-supervised learning is proposed. A full convolutional network F-VGG(Fully-Visual Geometry Group) is built as the FOE predictor, and the training label of the sample data is automatically generated through the fusion agent task, which realizes the end-to-end single-frame image FOE estimation. The experimental results show that the method has an average increase of 13.45% in FOE prediction accuracy, and an increase of 56.27% in detection speed, which is suitable for real-time applications.

Key words: self-supervised learning, motion parallax, fully convolutional network, Focus of Expansion, (FOE)

CLC Number:

TP391.4

Huo Zhihao, Jin Weidong, Tang Peng. Single-frame Image Motion Parallax Key Point Estimation Combined with Self-supervised Learning[J]. Journal of System Simulation, 2021, 33(11): 2753-2759.

References

[1] 胡燕花, 唐鹏, 金炜东, 等. 铁路视频序列的FOE的估计[J]. 计算机科学, 2018, 45(7): 226-229.
Hu Yanhua, Tang Peng, Jin Weidong, et al.Estimation of FOE of Railway Video Sequence[J]. Computer Science, 2018, 45(7): 226-229.
[2] 胡燕花. 接触网巡检图像的细节强化与场景重构[D].成都: 西南交通大学, 2018.
Hu Yanhua.Detail Enhancement and Scene Reconstruction of Catenary Inspection Images[D]. Chengdu: Southwest Jiaotong University, 2018.
[3] Tang P, Jin W.Railway Running Smoothness Monitoring based on Visual Motion Parallax[C]//2017 IEEE International Conference on Robotics and Biomimetics (ROBIO). Macao: IEEE, 2017: 2407-2412.
[4] Zhang L, Koch R.Vanishing Points Estimation and Line Classification in a Manhattan World[C]//Asian Conference on Computer Vision. Daejeon, Korea: Springer, Berlin, Heidelberg, 2012: 38-51.
[5] Geiger A, Lenz P, Urtasun R.Are We Ready for Autonomous Driving? the Kitti Vision Benchmark Suite[C]//2012 IEEE Conference on Computer Vision And Pattern Recognition. Providence, RI: IEEE, 2012: 3354-3361.
[6] Carlevaris-Bianco N, Ushani A K, Eustice R M.University of Michigan North Campus Long-term vision and Lidar Dataset[J]. The International Journal of Robotics Research (S0278-3649), 2016, 35(9): 1023-1035.
[7] 段旺旺. 面向监控视频的接触网设备检测及异常识别[D]. 成都: 西南交通大学, 2016.
Duan Wangwang.Catenary Equipment Detection and Abnormal Recognition for Surveillance Video[D]. Chengdu: Southwest Jiaotong University, 2016.
[8] 贺志明. 透视文本图像的灭点快速探测[J]. 上海工程技术大学报, 2012, 23(3): 229-232.
He Zhiming.Fast Detection of Vanishing Points in Perspective Text Images[J]. Journal of Shanghai University of Engineering Science, 2012, 23(3): 229-232.
[9] 谢文寒, 张祖勋, 张剑清. 一种新的基于灭点的相机标定方法[J]. 哈尔滨工业大学学报, 2003, 35(11): 1384-1387.
Xie Wenhan, Zhang Zuxun, Zhang Jianqing.A New Method of Camera Calibration based on Vanishing Point[J]. Journal of Harbin Institute of Technology, 2003, 35(11): 1384-1387.
[10] Li S, Wang L, Li J, et al.Image Classification Algorithm Based on Improved AlexNet[J]. Journal of Physics: Conference Series(S1742-6588), 2021, 1813(1): 012051.
[11] 王浩雨. 基于改进VGG16神经网络的异常检测模型研究[J]. 现代计算机, 2020(30): 3-7, 15.
Wang Haoyu.Research on Anomaly Detection Model Based on Improved VGG16 Neural Network[J]. Modern Computer, 2020(30): 3-7, 15.
[12] Kiliçarslan S, Celik M.RSigELU: A Nonlinear Activation Function for Deep Neural Networks[J]. Expert Systems with Applications (S0957-4174), 2021, 174: 114805.
[13] Li W.Analysis of Object Detection Performance Based on Faster R-CNN[J]. Journal of Physics: Conference Series. IOP Publishing (S1742-6588), 2021, 1827(1): 012085.
[14] 陈建廷, 向阳. 深度神经网络训练中梯度不稳定现象研究综述[J]. 软件学报, 2018, 29(7): 2071-2091.
Chen Jianting, Xiang Yang.A Review of Research on Gradient Instability in Deep Neural Network Training[J]. Journal of Software, 2018, 29(7): 2071-2091.
[15] Jing L, Tian Y.Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence (S0162-8828), 2021, 43(11): 4037-4058.
[16] 刘祥强, 李强, 何多魁, 等. 基于自监督学习的卷积神经网络在CT图像中的肝脏自动分割研究[J]. 科学技术创新, 2021(3): 78-79.
Liu Xiangqiang, Li Qiang, He Duokui, et al.Research on Automatic Liver Segmentation in CT Images Based on Self-supervised Learning Convolutional Neural Network[J]. Science and Technology Innovation, 2021(3): 78-79.
[17] Liu Y B, Zeng M, Meng Q H.D-vpnet: A Network for Real-time Dominant Vanishing Point Detection in Natural Scenes[J]. Neurocomputing(S0925-2312), 2020, 417: 432-440.