自动驾驶汽车感知系统仿真的研究及应用综述

doi:10.16182/j.issn1004731x.joss.22-FZ0921

摘要/Abstract

摘要：

随着科学技术的飞速进步，带有自动驾驶功能或辅助驾驶功能的车辆也开始涌入市场。交通事故的频发，使其安全问题成为人们关注的焦点。为测试自动驾驶汽车的感知系统，仿真技术提供了良好的平台。针对自动驾驶汽车感知系统的传感器仿真建模研究，从包含毫米波雷达、激光雷达、摄像头在内的单一传感器仿真、多传感器仿真、经典仿真平台等角度，回顾与梳理了现有研究，并总结了关于感知系统仿真建模的不足与发展趋势，为提升感知系统的感知效率、出行安全、促进自动驾驶汽车的发展助力。

关键词: 感知系统, 自动驾驶汽车, 传感器, 雷达

Abstract:

Following the rapid progress of science and technology, vehicles with autonomous driving or auxiliary driving function enter into vehicle market. However, in the past decade, traffic accidents still occurred frequently, and the safety of these functions become the focus. Simulation technology provides a good platform to test the perception system of autonomous vehicle. Focus on the sensor simulation modeling of autonomous vehicle perception system, from the perspective of single sensor simulation, multi-sensor simulation and classic simulation platform including millimeter wave radar, lidar and camera, the existing research are reviewed, and the shortcomings and development trends of simulation modeling of perception system are summarized. The perception efficiency, travel safety, and the development of autonomous vehicle is promoted.

Key words: perception system, autonomous vehicle, sensors, radar

中图分类号:

TP391.9

王若萱, 吴建平, 徐辉. 自动驾驶汽车感知系统仿真的研究及应用综述[J]. 系统仿真学报, 2022, 34(12): 2507-2521.

Ruoxuan Wang, Jianping Wu, Hui Xu. Overview of Research and Application on Autonomous Vehicle Oriented Perception System Simulation[J]. Journal of System Simulation, 2022, 34(12): 2507-2521.

图/表 3

表1

2013—2021年部分自动驾驶汽车事故

时间	国家	地点	事故描述
2021-07	美国	高速公路	特斯拉因未能识别出高速上静止的车辆和行人，导致事故车辆撞向高速公路左侧修理爆胎的人，被撞人当场死亡
2021-04	美国	高速公路	启用了AutoPilot自动驾驶系统行驶的特斯拉，由于在转弯处速度过快，导致事故车辆失控并撞向路旁树木引发车体自燃，车内2名乘客死亡
2019-12	美国	高速公路	一辆特斯拉未能在远处检测到静止的消防车并提前作出刹车动作，最终撞上静止的消防车车尾，导致司机死亡
2019-08	美国	高速公路	一辆福特皮卡向右变道，后方的特斯拉高速撞上了福特，导致福特副驾被甩出窗外
2019-03	美国	十字路口	汽车在驶入十字路口时撞上正在左转的货车，卡车车身相对较高，轿车车身相对较矮，毫米波雷达由于高度原因未能检测到卡车，导致轿车司机死亡
2018-04	日本	城市道路	位于特斯拉前方车辆因躲避人群变道避让，特斯拉系统检测到前方车辆离开之后判定需要加速(未能考虑到前方车辆变道的原因)，最终撞上人群导致一人死亡(司机瞌睡)
2018-03	美国	高速公路	特斯拉的Model X在高速公路上行驶时，由于未能判定出公路上的隔离墩，导致Model X直接撞上，司机当场死亡
2018-03	美国	城市道路	优步自动驾驶汽车因未能及时识别前方推着自行车的行人并做出反应，导致行人死亡
2018-01	美国	高速公路	消防车停在道路左肩，一辆特斯拉Model S跟在一辆轿车后在最左侧车道行驶，前方车辆变道之后特斯拉保持在原车道随即自动加速行驶，未预测到前方静止车辆，导致未能及时躲避前方停着的消防车
2017-11	美国	小巷	一辆卡车在倒车进入小巷子时不慎撞到自动驾驶的小型旅游巴士，小型巴士在即将发生碰撞时未有躲避动作
2016-05	美国	十字路口	特斯拉无人驾驶汽车因在强光照射下未能分辨出前方车辆底盘较高的白色货车，导致司机死亡
2016-01	中国	高速公路	中国京港澳高速河北邯郸段，一辆特斯拉无人驾驶汽车因未能分辨出前方与背景颜色相近的车辆，全速撞上了正在工作的道路清洁车辆，导致司机死亡

表1

表2

表3

仿真平台与其传感器仿真能力

软件/平台	发布时间	功能/优势	传感器仿真能力
CarSim	1996	属于整车动力学仿真软件，主要从整车角度进行仿真。内置相当数量的车辆数学模型，具有丰富的经验参数	与Matlab/Simulink联合仿真，对车辆配置感知传感器
PreScan	2011	可用于从基于模型的控制器设计(MIL)到利用软件在环(SIL)和硬件在环(HIL)系统进行的实时测试等应用。支持和Simulink，ROS，Autoware，python，FMI，C++的联合仿真	支持种类丰富的传感器，例如，V2X传感器，激光雷达、毫米波雷达、超声波雷达，单目和双目相机、鱼眼相机等
PanoSim	2014	具有复杂车辆动力学模型、汽车三维行驶环境模型、汽车行驶交通模型、车载环境传感模型、无线通信模型、GPS和数字地图模型。功能强大，包含Matlab/Simulink接口	基于几何模型与物理建模相结合理念建立了高精度的像机、雷达和无线通信模型
RightHook	2017	拥有一整套的工具链，包括RightWorld、RightWorldHD、RightWorldHIL。RightWorld提供了包含车辆、行人和自行车的确定性的智能交通仿真模型。RightWorldHD可实现对动力学、天气、时间变化的模拟，同时包括丰富的接口	包含摄像头、激光雷达毫米波雷达、IMU和GPS的模拟
CARLA	2017	开源模拟器，具有python接口。提供了用于创建场景的开源数字资源(包括城市布局、建筑以及车辆)以及部分搭建好的自动驾驶测试训练的场景	支持传感器和环境的灵活配置，可调节光照和天气。支持多摄像头、激光雷达、GPS等传感器的仿真
51Sim-One	2018	一体化的自动驾驶仿真与测试平台。该平台基于物理特性进行机理建模，具有高精度和实时仿真的特点	相比其他软件仿真能力更加强大。对于摄像头仿真提供语义分割图、深度图、2D/3等带注释的图像数据集，单目、广角、鱼眼等摄像头的仿真。可提供激光雷达点云原始数据、带标注点云数据等数据。同时也提供目标级毫米波雷达检测物数据

表3

参考文献 56

1	Bernsteiner S, Magosi Z, Lindvai-Soos D, et al. Radar Sensor Model for the Virtual Development Process[J]. ATZelektronik Worldwide(S2192-9092), 2015, 10(2): 46-52.
2	Cao P, Wachenfeld W, Winner H. Perception Sensor Modeling for Virtual Validation of Automated Driving[J]. IT-Information Technology(S2196-7032), 2015, 57(4): 243-251.
3	Stolz M, Nestlinger G. Fast Generic Sensor Models for Testing Highly Automated Vehicles in Simulation[J]. Elektrotechnik und Informationstechnik (S0932-383X), 2018, 135(4): 365-369.
4	Muckenhuber S, Holzer H, Rübsam J, et al. Object-Based Sensor Model for Virtual Testing of ADAS/Ad Functions[C]//2019 IEEE International Conference on Connected Vehicles and Expo (ICCVE). Graz, Austria: IEEE, 2019: 1-6.
5	Hanke T, Hirsenkorn N, Dehlink B, et al. Generic Architecture for Simulation of ADAS Sensors[C]// 2015 16th International Radar Symposium (IRS). Dresden: IEEE, 2015: 125-130.
6	Mesow L. Multisensorielle Datensimulation Im Fahrzeugumfeld FüR Die Bewertung Von Sensorfusionsalgorithmen[M]. Germany: Shaker, 2007.
7	Schuler K, Becker D, Wiesbeck W. Extraction of Virtual Scattering Centers of Vehicles by Ray-Tracing Simulations[J]. IEEE Transactions on Antennas and Propagation(S0018-926X), 2008, 56(11): 3543-3551.
8	Hammarstrand L, Svensson L, Sandblom F, et al. Extended Object Tracking Using a Radar Resolution Model[J]. IEEE Transactions on Aerospace and Electronic Systems(S0018-9251), 2012, 48(3): 2371-2386.
9	Hanke T, Schaermann A, Geiger M, et al. Generation and Validation of Virtual Point Cloud Data for Automated Driving Systems[C]//2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC). Yokohama: IEEE, 2017: 1-6.
10	Li X, Deng W, Zhang S, et al. Research on Millimeter Wave Radar Simulation Model for Intelligent Vehicle[J]. International Journal of Automotive Technology (S1229-9138), 2020, 21(2): 275-284.
11	Guo J, Deng W, Zhang S, et al. A Novel Method of Radar Modeling for Vehicle Intelligence[J]. SAE International Journal of Passenger Cars-Electronic and Electrical Systems(S1946-4614), 2016, 10(1): 50-56.
12	Zhu B, Sun Y, Zhao J, et al. Millimeter-Wave Radar in-the-Loop Testing for Intelligent Vehicles[J]. IEEE Transactions on Intelligent Transportation Systems (S1524-9050), 2022, 23(8): 11126-11136.
13	Dudek M, Wahl R, Kissinger D, et al. Millimeter Wave FMCW Radar System Simulations Including a 3D Ray Tracing Channel Simulator[C]//2010 Asia-Pacific Microwave Conference Proceedings(APMC). Yokohama: IEEE, 2010: 1665-1668.
14	Schlager B, Muckenhuber S, Schmidt S, et al. State-of-the-Art Sensor Models for Virtual Testing of Advanced Driver Assistance Systems/Autonomous Driving Functions[J]. SAE International Journal of Connected and Automated Vehicles(S2574-075X), 2020, 3(3): 233-261. DOI:10.4271/12-03-03-0018 .
15	Hirsenkorn N, Subkowski P, Hanke T, et al. A Ray Launching Approach for Modeling an FMCW Radar System[C]//2017 18th International Radar Symposium (IRS). Prague: IEEE, 2017: 1-10.
16	Maier F, Makkapati V, Horn M. Environment Perception Simulation for Radar Stimulation in Automated Driving Function Testing[J]. Elektrotechnik und Informationstechnik(S0932-383X), 2018, 135(4): 309-315.
17	Suzuki Y, Sugiura Y, Shimamura T, et al. Model-Based Vehicle Position Estimation Using Millimeter Wave Radar[J]. International Journal of Future Computer and Communication(S2010-3751), 2019, 8(3): 94-98.
18	Song Y, Wang Y, Li Y. Radar Data Simulation Using Deep Generative Networks[J]. The Journal of Engineering(S2051-3305), 2019(20): 6699-6702.
19	Zhao J, Li Y, Zhu B, et al. Method and Applications of Lidar Modeling for Virtual Testing of Intelligent Vehicles[J]. IEEE Transactions on Intelligent Transportation Systems(S1524-9050), 2021, 22(5): 2990-3000.
20	Khatry R, Thompson A. Camera and Lidar Sensor Models for Autonomous Vehicles[R]. Teddington: National Physical Laboratory, 2021.
21	Byeon M, Yoon S W. Analysis of Automotive Lidar Sensor Model Considering Scattering Effects in Regional Rain Environments[J]. IEEE Access(S2169-3536), 2020, 8: 102669-102679.
22	Peinecke N, Lueken T, Korn B R. Lidar Simulation Using Graphics Hardware Acceleration[C]//2008 IEEE/AIAA 27th Digital Avionics Systems Conference. St. Paul, MN: IEEE, 2008.
23	Wang S, Heinrich S, Wang M, et al. Shader-Based Sensor Simulation for Autonomous Car Testing[C]//2012 15th International IEEE Conference on Intelligent Transportation Systems. Anchorage, AK: IEEE, 2012: 224-229.
24	OBrien M E, Fouche D G. Simulation of 3D Laser Radar Systems[J]. Lincoln Laboratory Journal, 2005, 15(1): 37-60.
25	Manivasagam S, Wang S, Wong K, et al. LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Seattle, WA: IEEE, 2020: 11164-11173.
26	Fang J, Zhou D, Yan F, et al. Augmented LiDAR Simulator for Autonomous Driving[J]. IEEE Robotics and Automation Letters(S2377-3766), 2020, 5(2): 1931-1938.
27	Goodin C, Kala R, Carrrillo A, et al. Sensor Modeling for the Virtual Autonomous Navigation Environment[C]//2009 IEEE Sensors Conference. Christchurch: IEEE, 2009: 1588-1592. DOI:10.1109/ICSENS.2009.5398491 .
28	Dosovitskiy A, Ros G, Codevilla F, et al. CARLA: An Open Urban Driving Simulator[C]// Conference on robot learning. California: Proceedings of Machine Learning Research, 2017: 1-16.
29	Shah S, Dey D, Lovett C, et al. AirSim: High-fidelity Visual and Physical Simulation for Autonomous Vehicles[C]//Field and Service Robotics. Zurich: Springer, Cham, 2018: 621-635.
30	Jiang H, Tian Q, Farrell J, et al. Learning the Image Processing Pipeline[J]. IEEE Transaction on Image Processing(S1057-7149), 2017, 26(10): 5032-5042.
31	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks[J]. Communications of the ACM(S1557-7317), 2020, 63(11): 139-144.
32	Sturm P, Ramalingam S, Tardif J P, et al. Camera Models and Fundamental Concepts Used in Geometric Computer Vision[J]. Foundations and Trends in Computer Graphics and Vision(S1572-2759), 2011, 6(1/2): 1-183.
33	Santana-Cedrés D, Gomez L, Alemán Flores M, et al. Estimation of the Lens Distortion Model by Minimizing a Line Reprojection Error[J]. IEEE Sensors Journal(S1530-437X), 2017, 17(9): 2848-2855.
34	薛艺舟. 面向自动驾驶的激光雷达与相机联合标定方法研究[D]. 北京: 北京建筑大学, 2021. DOI:10.26943/d.cnki.gbjzc.2021.000083 .
	Xue Yizhou. Research on the Joint Calibration Method of LiDAR and Camera for Autonomous Driving[D]. Beijing: Beijing University of Civil Engineering and Architecture, 2021. DOI:10.26943/d.cnki.gbjzc.2021.000083 .
35	张姗. 智能驾驶车辆环境感知系统建模与仿真[D]. 长春: 长春大学, 2021. DOI:10.27912/d.cnki.gcdcx.2021.000106 .
	Zhang Shan. Modeling and Simulation of Environment Awareness System for Intelligent Driving Vehicles[D]. Changchun: Changchun University, 2021. DOI:10.27912/d.cnki.gcdcx.2021.000106 .
36	王瑞. 基于车载的多相机联合标定[D]. 西安: 西安电子科技大学, 2017.
	Wang Rui. Multi-Camer a Joint Calibration Based on Vehicle[D]. Xiʹan: Xidian University, 2017.
37	舒娜. 摄像机标定方法的研究[D]. 南京: 南京理工大学, 2014.
	Shu Na. Research on Camera Calibration Method[D]. Nanjing: Nanjing University of Science and Technology, 2014.
38	Elmquist A, Negrut D. Modeling Cameras for Autonomous Vehicle and Robot Simulation: An Overview[J]. IEEE Sensors Journal(S1530-437X), 2021, 21(22): 25547-25560.
39	Walch A, Luksch C, Szabo A, et al. Lens Flare Prediction Based on Measurements with Real-time Visualization[J] The Visual Computer(S0178-2789), 2018, 34(9): 1155-1164.
40	Zheng Q, Zheng C. NeuroLens: Data-driven Camera Lens Simulation Using Neural Networks[J]. Computer Graphics Forum(S1467-8659), 2017, 36(8): 390-401.
41	Bisagno N, Conci N. Virtual Camera Nodeling for Multi-view Simulation of Surveillance Scenes[C]//2018 26th European Signal Processing Conference (EUSIPCO). Rome: IEEE, 2018: 2170-2174.
42	Pharr M, Jakob W, Humphreys G. Physically Based Rendering: From Theory to Implementation[M]. San Mateo, CA, USA: Morgan Kaufmann, 2016.
43	Afifi M, Brown M S. Deep White-balance Editing[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA: IEEE, 2020: 1397-1406.
44	Liu C, Szeliski R, Kang S B, et al. Automatic Estimation and Removal of Noise from a Single Image[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(S0162-8828), 2008, 30(2): 299-314.
45	Rosenberger P, Holder M, Huch S, et al. Benchmarking and Functional Decomposition of Automotive Lidar Sensor Models[C]//2019 IEEE Intelligent Vehicles Symposium (IV). Paris: IEEE, 2019: 632-639.
46	齐翔. 基于多传感器信息融合技术的汽车行驶状态估计[D]. 镇江: 江苏大学, 2019.
	Qi Xiang. State Estimation of Vehicle Based on Multi-sensors Information Fusion[D]. Zhenjiang: Jiangsu University, 2019.
47	刘敏. 基于自动驾驶车载毫米波雷达间相互干扰的研究[D]. 重庆: 重庆理工大学, 2021.
	Liu Min. Research on Mutual Interference between Millimeter Wave Radars for Autonomous Vehicles[D]. Chongqing: Chongqing University of Technology, 2021.
48	毛雪松, 雷闰龙, 黄少伟, 等. 道路环境中激光雷达互干扰的仿真研究[J]. 系统仿真学报, 2022, 34(4): 891-900.
	Mao Xuesong, Lei Runlong, Huang Shaowei, et al. Simulation on Mutual Interference of Laser Radar in Road Environments[J]. Journal of System Simulation, 2022, 34(4): 891-900.
49	刘晶郁, 马辉, 张学文, 等. 自动驾驶虚拟仿真测试技术研究进展[J]. 中国科技论文, 2021, 16(6): 571-577, 584.
	Liu Jingyu, Ma Hui, Zhang Xuewen, et al. Research and Development of Autonomous Driving Virtual Simulation Technology[J]. China Science Paper, 2021, 16(6): 571-577, 584.
50	清华大学－剑桥大学－麻省理工学院低碳能源大学联盟未来交通研究中心. 交通仿真软件FLOWSIM[EB/OL]. (2021-04-03)[2021-06-01]. .
	Tsinghua University-University of Cambridge-Massachusetts Institute of Technology Low Carbon Energy University Alliance Future Transport Research Center. Traffic Simulation Software: FLOWSIM[EB/OL]. (2021-04-03)[2021-06-01]. .
51	毕锟, 范英, 候峙朴, 等. 基于Carsim和Simulink联合仿真的汽车操纵稳定性评价[J]. 河北工程大学学报(自然科学版), 2022, 39(2): 106-112.
	Bi Kun, Fan Ying, Hou Zhipu, et al. Vehicle Handling Stability Evaluation Based on CarSim and Simulink Co-Simulation[J]. Journal of Hebei University of Engineering (Natural Science Edition), 2022, 39(2): 106-112.
52	朱鹏飞, 梁威, 丁文政, 等. 基于PreScan的自动泊车仿真研究[J]. 计算机与数字工程, 2021, 49(1): 122-125.
	Zhu Pengfei, Liang Wei, Ding Wenzheng, et al. Research on Automatic Parking Simulation Based on PreScan[J]. Computer & Digital Engineering, 2021, 49(1): 122-125.
53	郭路兵, 梅志千, 贺勇. 基于PreScan软件的交叉路口防碰撞控制系统仿真[J]. 机械与电子, 2014(2): 22-26.
	Guo Lubing, Mei Zhiqian, He Yong. Simulation of Crossroads Anti‑collision Control System Based on PreScan[J]. Machinery and Electronics, 2014(2): 22-26.
54	王成康, 黄李波. 基于CARLA的驾驶仿真平台搭建[J]. 佳木斯大学学报(自然科学版), 2021, 39(3): 27-29.
	Wang Chengkang, Huang Libo. Construction of Driving Simulation Platform Based on CARLA[J]. Journal of Jiamusi University(Natural Science Edition), 2021, 39(3): 27-29.
55	Brekke Å, Vatsendvik F, Lindseth F. Multimodal 3D Object Detection from Simulated Pretraining[C]//Symposium of the Norwegian AI Society. Trondheim: Springer, Cham, 2019: 102-113.
56	Zapridou E, Bartocci E, Katsaros P. Runtime Verification of Autonomous Driving System in CARLA[C]//International Conference on Runtime Verification. Los Angeles, CA: Springer, Cham, 2020: 172-183.

畸变种类	产生原因
径向畸变	相机组装过程中，透镜与芯片不平行引起光心与几何中心不一致造成的畸变
切向畸变	由于制作工艺缺陷引起透镜形状改变造成的畸变
色差	波长相关折射引起的颜色位置偏移
渐晕	透镜自身原因，光线离轴越远，光强度越弱
光斑	强光的多径反射产生的鬼影
景深模糊	超出镜头纵深清晰范围的部分出现模糊，受光圈、焦距、物距影响

[1]	张会林, 金玉洁, 杨海马. ANFIS优化磁链滑模观测器的PMSM无传感器控制[J]. 系统仿真学报, 2022, 34(8): 1682-1690.
[2]	于帆, 罗东琦, 司宾强, 朱纪洪. 可交互式相控阵雷达DOA估计仿真系统设计[J]. 系统仿真学报, 2022, 34(5): 964-977.
[3]	康旭, 张晓峰. 基于生成对抗神经网络的雷达遥感数据增广方法[J]. 系统仿真学报, 2022, 34(4): 920-927.
[4]	毛雪松, 雷闰龙, 黄少伟, 毛雪涛. 道路环境中激光雷达互干扰的仿真研究[J]. 系统仿真学报, 2022, 34(4): 891-900.
[5]	吴静, 徐志明, 艾小锋, 赵锋, 肖顺平. 组网雷达融合处理组件化设计与仿真[J]. 系统仿真学报, 2022, 34(2): 409-418.
[6]	娄嘉冀, 马静, 李晓威, 赵越, 宋友斌. 微波雷达高精度跟踪环路建模与仿真技术研究[J]. 系统仿真学报, 2022, 34(12): 2619-2628.
[7]	徐公国, 蔡利兵, 杜配冰, 刘钰. 复杂环境下多探测传感器协同定位优化布站建模仿真研究[J]. 系统仿真学报, 2022, 34(10): 2171-2180.
[8]	袁飞, 李建宏, 银皓, 王玉皞, 洪升. 一种舰船目标SAR回波的快速仿真方法[J]. 系统仿真学报, 2022, 34(1): 11-19.
[9]	李纲, 张彩霞, 胡绍林, 王向东, 郭静. 面向桥梁结构健康监测的WSN分簇路由协议[J]. 系统仿真学报, 2022, 34(1): 62-69.
[10]	黄星源, 李岩屹. 基于双Q学习算法的干扰资源分配策略[J]. 系统仿真学报, 2021, 33(8): 1801-1808.
[11]	冯新扬, 邵超. 跨卷积网络特征融合的SAR图像目标识别[J]. 系统仿真学报, 2021, 33(3): 554-561.
[12]	王振东, 谢华茂, 胡中栋, 李大海, 王俊岭. 改进花朵授粉算法的无线传感器网络部署优化[J]. 系统仿真学报, 2021, 33(3): 645-656.
[13]	刁桂杰, 倪虹, 杨亮, 巩宁波, 郭姣. 宽带相控阵雷达海面场景散射回波模拟方法研究[J]. 系统仿真学报, 2021, 33(2): 401-408.
[14]	黄颖坤, 金炜东, 颜康, 朱劼昊. 基于距离特征的雷达辐射源信号识别方法[J]. 系统仿真学报, 2021, 33(12): 2959-2966.
[15]	王蕊, 李向阳, 张志利, 马红光. 基于多干扰机协同的航迹欺骗建模方法[J]. 系统仿真学报, 2021, 33(12): 2992-3000.