融合自监督学习的单帧图像运动视差关键点估计

doi:10.16182/j.issn1004731x.joss.21-FZ0708

系统仿真学报 ›› 2021, Vol. 33 ›› Issue (11): 2753-2759.doi: 10.16182/j.issn1004731x.joss.21-FZ0708

• 快报/短文 • 上一篇

融合自监督学习的单帧图像运动视差关键点估计

霍志浩¹, 金炜东^1,2,*, 唐鹏¹

1.西南交通大学电气工程学院,四川成都 611756;
2.南宁学院中国-东盟综合交通国际联合实验室,广西南宁 530200

收稿日期:2021-04-30 修回日期:2021-08-18 出版日期:2021-11-18 发布日期:2021-11-17
通讯作者: 金炜东(1959-),男,博士,博导,教授,研究方向为智能信息处理。E-mail：wdjin@home.swjtu.edu.cn
作者简介:霍志浩(1996-),男,硕士生,研究方向为深度学习。E-mail：171937824@qq.com
基金资助:
国家重点研发计划(2016YFB200401-102F)

Single-frame Image Motion Parallax Key Point Estimation Combined with Self-supervised Learning

Huo Zhihao¹, Jin Weidong^1,2,*, Tang Peng¹

1. School of Electric Engineering, Southwest Jiaotong University, Chengdu 611576, China;
2. China-ASEAN International Joint Laboratory of Integrated Transport, Nanning University, Guangxi 530200, China

Received:2021-04-30 Revised:2021-08-18 Online:2021-11-18 Published:2021-11-17

摘要/Abstract

摘要： 运动视差的关键点(Focus of Expansion,FOE)是铁路接触网视频巡检的重要参数,但当前计算FOE的方法需多帧图像匹配估计,时间复杂度高。针对单帧图像FOE估计问题,结合自监督学习思想,提出了一种融合自监督学习的单帧图像FOE估计算法。搭建了全卷积网络F-VGG (Fully-Visual Geometry Group)作为FOE的预测器,通过融合代理任务自动生成样本数据的训练标签,实现了端到端的单帧图像FOE估计。实验结果表明：该方法在FOE预测精度上平均提升13.45%,检测速度提升56.27%,适于实时应用。

关键词: 自监督学习, 运动视差, 全卷积网络, FOE (Focus of Expansion)

Abstract: The motion parallax key point FOE (Focus of Expansion) is an important parameter of railway catenary video inspection. The current method of calculating FOE requires multi-frame image matching estimation, which has high time complexity. Aiming at the single-frame image FOE estimation, a single-frame image FOE estimation algorithm fused with self-supervised learning is proposed. A full convolutional network F-VGG(Fully-Visual Geometry Group) is built as the FOE predictor, and the training label of the sample data is automatically generated through the fusion agent task, which realizes the end-to-end single-frame image FOE estimation. The experimental results show that the method has an average increase of 13.45% in FOE prediction accuracy, and an increase of 56.27% in detection speed, which is suitable for real-time applications.

Key words: self-supervised learning, motion parallax, fully convolutional network, Focus of Expansion, (FOE)

中图分类号:

TP391.4

霍志浩, 金炜东, 唐鹏. 融合自监督学习的单帧图像运动视差关键点估计[J]. 系统仿真学报, 2021, 33(11): 2753-2759.

Huo Zhihao, Jin Weidong, Tang Peng. Single-frame Image Motion Parallax Key Point Estimation Combined with Self-supervised Learning[J]. Journal of System Simulation, 2021, 33(11): 2753-2759.

参考文献

[1] 胡燕花, 唐鹏, 金炜东, 等. 铁路视频序列的FOE的估计[J]. 计算机科学, 2018, 45(7): 226-229.
Hu Yanhua, Tang Peng, Jin Weidong, et al.Estimation of FOE of Railway Video Sequence[J]. Computer Science, 2018, 45(7): 226-229.
[2] 胡燕花. 接触网巡检图像的细节强化与场景重构[D].成都: 西南交通大学, 2018.
Hu Yanhua.Detail Enhancement and Scene Reconstruction of Catenary Inspection Images[D]. Chengdu: Southwest Jiaotong University, 2018.
[3] Tang P, Jin W.Railway Running Smoothness Monitoring based on Visual Motion Parallax[C]//2017 IEEE International Conference on Robotics and Biomimetics (ROBIO). Macao: IEEE, 2017: 2407-2412.
[4] Zhang L, Koch R.Vanishing Points Estimation and Line Classification in a Manhattan World[C]//Asian Conference on Computer Vision. Daejeon, Korea: Springer, Berlin, Heidelberg, 2012: 38-51.
[5] Geiger A, Lenz P, Urtasun R.Are We Ready for Autonomous Driving? the Kitti Vision Benchmark Suite[C]//2012 IEEE Conference on Computer Vision And Pattern Recognition. Providence, RI: IEEE, 2012: 3354-3361.
[6] Carlevaris-Bianco N, Ushani A K, Eustice R M.University of Michigan North Campus Long-term vision and Lidar Dataset[J]. The International Journal of Robotics Research (S0278-3649), 2016, 35(9): 1023-1035.
[7] 段旺旺. 面向监控视频的接触网设备检测及异常识别[D]. 成都: 西南交通大学, 2016.
Duan Wangwang.Catenary Equipment Detection and Abnormal Recognition for Surveillance Video[D]. Chengdu: Southwest Jiaotong University, 2016.
[8] 贺志明. 透视文本图像的灭点快速探测[J]. 上海工程技术大学报, 2012, 23(3): 229-232.
He Zhiming.Fast Detection of Vanishing Points in Perspective Text Images[J]. Journal of Shanghai University of Engineering Science, 2012, 23(3): 229-232.
[9] 谢文寒, 张祖勋, 张剑清. 一种新的基于灭点的相机标定方法[J]. 哈尔滨工业大学学报, 2003, 35(11): 1384-1387.
Xie Wenhan, Zhang Zuxun, Zhang Jianqing.A New Method of Camera Calibration based on Vanishing Point[J]. Journal of Harbin Institute of Technology, 2003, 35(11): 1384-1387.
[10] Li S, Wang L, Li J, et al.Image Classification Algorithm Based on Improved AlexNet[J]. Journal of Physics: Conference Series(S1742-6588), 2021, 1813(1): 012051.
[11] 王浩雨. 基于改进VGG16神经网络的异常检测模型研究[J]. 现代计算机, 2020(30): 3-7, 15.
Wang Haoyu.Research on Anomaly Detection Model Based on Improved VGG16 Neural Network[J]. Modern Computer, 2020(30): 3-7, 15.
[12] Kiliçarslan S, Celik M.RSigELU: A Nonlinear Activation Function for Deep Neural Networks[J]. Expert Systems with Applications (S0957-4174), 2021, 174: 114805.
[13] Li W.Analysis of Object Detection Performance Based on Faster R-CNN[J]. Journal of Physics: Conference Series. IOP Publishing (S1742-6588), 2021, 1827(1): 012085.
[14] 陈建廷, 向阳. 深度神经网络训练中梯度不稳定现象研究综述[J]. 软件学报, 2018, 29(7): 2071-2091.
Chen Jianting, Xiang Yang.A Review of Research on Gradient Instability in Deep Neural Network Training[J]. Journal of Software, 2018, 29(7): 2071-2091.
[15] Jing L, Tian Y.Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence (S0162-8828), 2021, 43(11): 4037-4058.
[16] 刘祥强, 李强, 何多魁, 等. 基于自监督学习的卷积神经网络在CT图像中的肝脏自动分割研究[J]. 科学技术创新, 2021(3): 78-79.
Liu Xiangqiang, Li Qiang, He Duokui, et al.Research on Automatic Liver Segmentation in CT Images Based on Self-supervised Learning Convolutional Neural Network[J]. Science and Technology Innovation, 2021(3): 78-79.
[17] Liu Y B, Zeng M, Meng Q H.D-vpnet: A Network for Real-time Dominant Vanishing Point Detection in Natural Scenes[J]. Neurocomputing(S0925-2312), 2020, 417: 432-440.

融合自监督学习的单帧图像运动视差关键点估计

Single-frame Image Motion Parallax Key Point Estimation Combined with Self-supervised Learning

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	周培培, 侯幸林. 一种用于图像融合的无监督深度神经网络[J]. 系统仿真学报, 2022, 34(6): 1267-1274.
[2]	曹建芳, 贾一鸣, 闫敏敏, 田晓东. 稳定增强生成对抗网络在壁画的超分辨率重建[J]. 系统仿真学报, 2022, 34(5): 1076-1089.
[3]	冯开团, 袁杰. 基于改进注水算法的离散车间任务分配问题研究[J]. 系统仿真学报, 2022, 34(4): 768-776.
[4]	宁小娟, 李洁茹, 高凡, 王映辉. 基于最佳几何约束和RANSAC的特征匹配算法[J]. 系统仿真学报, 2022, 34(4): 727-734.
[5]	杨敬辉, 刘德康, 杜万和, 邢立宁. 基于图像特征的双目测距系统的研究[J]. 系统仿真学报, 2022, 34(3): 624-632.
[6]	王语童, 马世伟, 杨元睿, 陈超宇. 多机器人组合最大覆盖面积寻优及预测方法[J]. 系统仿真学报, 2022, 34(1): 86-92.
[7]	刘茂山, 纪志成, 王艳, 王建锋. 核块对角表达子空间聚类及收敛性分析[J]. 系统仿真学报, 2021, 33(11): 2533-2544.
[8]	胡蕾, 王足根, 陈田, 张永梅. 一种改进的SRGAN红外图像超分辨率重建算法[J]. 系统仿真学报, 2021, 33(9): 2109-2118.
[9]	李新利, 邹昌铭, 杨国田, 刘禾. 基于生成式对抗网络的发票图像超分辨率研究[J]. 系统仿真学报, 2021, 33(6): 1307-1314.
[10]	柳有权, 裴雪, 李婉, 刘正雄. 基于邻近目标置信度评估的视觉目标跟踪与定位[J]. 系统仿真学报, 2020, 32(7): 1294-1300.
[11]	汤雯洁, 李若桐, 邓雪松, 司伟鑫, 王琼. ICP优化下的B样条四维CT图像弹性配准方法[J]. 系统仿真学报, 2020, 32(7): 1301-1311.
[12]	王素琴, 张峰, 高宇豆, 石敏. 基于图像序列的学习表情识别[J]. 系统仿真学报, 2020, 32(7): 1322-1330.
[13]	张惠娟, 郭欣琪, 王冬青, 贾金原. 基于DR预测的大规模Web3D场景预加载机制[J]. 系统仿真学报, 2020, 32(7): 1341-1348.
[14]	钱文华, 徐丹, 徐瑾, 何磊, 张波. 东巴画艺术风格绘制[J]. 系统仿真学报, 2020, 32(7): 1349-1359.
[15]	刘佳哲, 陈纯毅, 胡小娟, 梁伟东, 邢琦玮, 杨华民. 面向手机的云端VR三维场景立体显示与交互框架[J]. 系统仿真学报, 2020, 32(7): 1360-1374.