基于单目深度估计的低功耗视觉里程计

doi:10.16182/j.issn1004731x.joss.21-FZ0863

系统仿真学报 ›› 2021, Vol. 33 ›› Issue (12): 3001-3011.doi: 10.16182/j.issn1004731x.joss.21-FZ0863

• 仿真模型/系统置信度评估技术 • 上一篇下一篇

基于单目深度估计的低功耗视觉里程计

马榕¹, 陈秋瑞¹, 张晗¹, 梅铮¹, 王锐², 魏伟³

1.北京仿真中心航天系统仿真重点实验室,北京 100854;
2.北京航空航天大学,北京 100191;
3.空军装备部驻北京地区第四军事代表室,北京 100041

收稿日期:2021-06-09 修回日期:2021-08-25 出版日期:2021-12-18 发布日期:2022-01-13
通讯作者: 张晗(1978-),女,硕士,高工,研究方向为高效能计算与仿真。E-mail：xia_mei2000@163.com
作者简介:马榕(1995-),女,硕士,工程师,研究方向为计算机视觉与仿真。E-mail：marong_xidian@163.com

Low Power Visual Odometry Technology Based on Monocular Depth Estimation

Ma Rong¹, Chen Qiurui¹, Zhang Han¹, Mei Zheng¹, Wang Rui², Wei Wei³

1. Science and Technology on Special System Simulation Laboratory, Beijing 100854, China;
2. BeiHang University, Beijing 100191, China;
3. The Fourth Military Representative Office of the Air Force Armament Department in Beijing, Beijing 100041, China

Received:2021-06-09 Revised:2021-08-25 Online:2021-12-18 Published:2022-01-13

摘要/Abstract

摘要： 随着人工智能、精密机械和计算技术的发展,微小型无人系统在未来战场上将会扮演重要的角色。为解决单目视觉里程计尺度缺失以及微型机器人自身体积和载荷限制带来的功耗问题,引入单目深度估计技术,构建了一个低视角数据集,通过搭建卷积神经网络从单张图像中预测深度信息,对神经网络模型进行结构优化,将深度估计与单目视觉里程计融合并部署到JetsonNano平台。实验表明,融合后的单目视觉里程计能够在特定环境下恢复尺度信息,在JetsonNano上的功耗能够保持在较低水平,可为微型无人系统在未来战场上的隐蔽化、轻量化部署提供一定的研究基础。

关键词: 单目视觉里程计, 单目深度估计, 卷积神经网络, 神经网络结构优化

Abstract: With the development of artificial intelligence, precision machinery and computing technology, micro-unmanned system will play an important role in the future battlefield. To solve the lack of monocular visual odometry scale, micro robot power consumption and load limits, the monocular depth estimation technology is introduced and a low view dataset is collected. A convolutional neural network to predict depth information from a single image is built, and the structure of neural network model is optimized. The depth estimation with monocular visual odometry are combined and deployed on JetsonNano. Experiments show that the combined monocular visual odometry can recover scale information in a specific environment, and the power consumption on Jetson Nano can be kept a low level, which can provide some research basis for the concealable and lightweight deployment of micro-unmanned system in the future battlefield.

Key words: monocular visual odometry, monocular depth estimation, convolution neural network, structure optimization

中图分类号:

TP391.9

马榕, 陈秋瑞, 张晗, 梅铮, 王锐, 魏伟. 基于单目深度估计的低功耗视觉里程计[J]. 系统仿真学报, 2021, 33(12): 3001-3011.

Ma Rong, Chen Qiurui, Zhang Han, Mei Zheng, Wang Rui, Wei Wei. Low Power Visual Odometry Technology Based on Monocular Depth Estimation[J]. Journal of System Simulation, 2021, 33(12): 3001-3011.

参考文献

[1] Laina I, Rupprecht C, Belagiannis V, et al.Deeper Depth Prediction with Fully Convolutional Residual Networks[C]// Fourth International Conference on 3d Vision. Piscataway, NY: IEEE, 2016.
[2] Godard C, Mac A O, Brostow G.Unsupervised Monocular Depth Estimation with Left-right Consistency[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NY: IEEE, 2016.
[3] Engel J, Koltun V, Cremers D.Direct Sparse Odometry[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence (S1939-3539), 2018, 40(3): 611-625.
[4] Mur-Artal R, Montiel J M M, Tardos J D. ORB-SLAM: A Versatile and Accurate Monocular SLAM System[J]. IEEE Transactions on Robotics (S1941-0468), 2017, 31(5): 1147-1163.
[5] Christian F, Matia P, Davide S.SVO: Fast Semi-direct Monocular Visual Odometry[C]// IEEE International Conference on Robotics and Automation. Piscataway, NY: IEEE, 2014.
[6] Leutenegger S, Lynen S, Bosse M, et al.Keyframe-based Visual-inertial Odometry Using Nonlinear Optimization[J]. International Journal of Robotics Research (S0278-3649), 2014, 34(3): 314-334.
[7] Loo S Y, Amiri A J, Mashohor S, et al.CNN-SVO: Improving the Mapping in Semi-Direct Visual Odometry Using Single-Image Depth Prediction[C]// 2019 International Conference on Robotics and Automation (ICRA). Piscataway, NY: IEEE, 2019.
[8] Silberman N, Hoiem D, Kohli P, et al.R.: Indoor Segmentation and Support Inference from RGBD Images[C]// Berlin, German: ECCV, 2012.
[9] Chollet F.Xception: Deep Learning with Depthwise Separable Convolutions[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NY: IEEE, 2017.
[10] Howard A G, Zhu M, Chen B, et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications[J]. arXiv:1704.04861, 2017.
[11] He K, Zhang X, Ren S, et al.Deep Residual Learning for Image Recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NY: IEEE, 2016.
[12] Jin H, Han M, Dong W, et al.From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NY: IEEE, 2019.
[13] Yang T J, Howard A, Chen B, et al.NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications[C]// Computer Vision - ECCV 2018. Munich, Germany: ECCV, 2018.
[14] George V A, Carlos H B.Video-based, Real-time Multi-view Stereo[J]. Image & Vision Computing (S0262-8856), 2011, 29(7): 434-441.
[15] Forster C, Zhang Z, Gassner M, et al.SVO: Semi-direct Visual Odometry for Monocular and Muticameras Systems[J]. IEEE Transactions on Robotics (S1941-0468), 2017, 33(2): 249-265.
[16] Mur-Artal R, Juan Tardós D.ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras[C]// Piscataway, NY: IEEE Trans. Robot, 2017.

基于单目深度估计的低功耗视觉里程计

Low Power Visual Odometry Technology Based on Monocular Depth Estimation

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张立峰, 王会忍. 基于卷积神经网络及有限元仿真的电容层析成像图像重建[J]. 系统仿真学报, 2022, 34(4): 712-718.
[2]	敖邦乾, 杨莎, 令狐金卿, 叶振环. 基于级联神经网络疲劳驾驶检测系统设计[J]. 系统仿真学报, 2022, 34(2): 323-333.
[3]	赵瑛, 陆耀, 张健, 梁启弟, 龙炜. 基于深度神经网络的多视角人体动作识别[J]. 系统仿真学报, 2021, 33(5): 1019-1030.
[4]	仝卫国, 庞雪纯, 朱赓宏. 基于卷积神经网络的气液两相流流型识别方法[J]. 系统仿真学报, 2021, 33(4): 883-891.
[5]	黄欣, 方钰, 顾梦丹. 基于卷积神经网络的X线胸片疾病分类研究[J]. 系统仿真学报, 2020, 32(6): 1188-1194.
[6]	樊名鲁, 王艳, 纪志成. 基于特征生成的轴承不均衡数据故障诊断[J]. 系统仿真学报, 2020, 32(12): 2438-2448.
[7]	王红军, 黎邹邹, 邹湘军. 基于Adaboost与CNN的木材表面缺陷检测[J]. 系统仿真学报, 2019, 31(8): 1636-1645.
[8]	卢裕秋, 孙金玉, 马世伟. 基于深度卷积神经网络的运动目标检测方法[J]. 系统仿真学报, 2019, 31(11): 2275-2280.
[9]	石祥滨, 钟健, 刘翠微, 刘芳, 张德园. 小样本高光谱遥感图像深度学习方法[J]. 系统仿真学报, 2018, 30(7): 2744-2752.
[10]	夏旻, 申茂阳, 王舰锋, 王阳光. 基于卷积神经网络的卫星云图云量计算[J]. 系统仿真学报, 2018, 30(5): 1623-1630.
[11]	廖鹰, 易卓, 胡晓峰, 田园, 陶九阳. 基于三维卷积神经网络的战场聚集行为预测[J]. 系统仿真学报, 2018, 30(3): 801-808.
[12]	吴昀璞, 金炜东, 黄颖坤. 基于多域融合CNN的高速列车转向架故障检测[J]. 系统仿真学报, 2018, 30(11): 4492-4497.
[13]	高尚兵, 颉正, 潘志庚, 覃方哲, 李锐. 一种新的路面裂缝自动检测算法[J]. 系统仿真学报, 2017, 29(9): 2009-2015.
[14]	王松, 刘复昌, 黄骥, 许威威, 董洪伟. 基于卷积神经网络的深度图姿态估计算法研究[J]. 系统仿真学报, 2017, 29(11): 2618-2623.
[15]	朱丰, 胡晓峰, 贺筱媛, 孔亦思, 杨璐. 一种基于CNN的样本不足战场包围态势认知方法[J]. 系统仿真学报, 2017, 29(10): 2291-2300.