An Intelligent Tracking Control Method for Unmanned Vehicles with Time-varying Disturbances

doi:10.16182/j.issn1004731x.joss.23-1494

Abstract

Abstract:

An intelligent policy iteration tracking control method is proposed for the tracking control problem with bounded time-varying disturbances. An adaptive disturbance compensator is designed to counteract the bounded disturbance and guarantee the validity of the Hamilton-Jacobi-Bellman (HJB) equation. An identifier network is proposed to estimate the unknown vehicle dynamics, and a new HJB equation is derived using the reconstructed identifier tracking error. An online optimal tracking control strategy for unmanned vehicles is obtained in the state of identifier estimation with the assistance of actor-critic network. Based on Lyapunov theory, it is demonstrated that the identifier tracking error, identifier approximation error and neural network weight errors are all semi-globally uniformly ultimately bounded, and the unmanned vehicle can achieve ideal tracking performance. Simulation results indicate that with a disturbance upper-bounded by 10.331 1 N, the tracking error converges to at least 0.054 8 m which exhibits better anti-interference performance. Compared with sliding mode control, tracking accuracy is improved by 40% and control cost is reduced by 22%.

Key words: unmanned vehicles, time-varying disturbance, identifier network, strategy learning, tracking control

CLC Number:

TP391.9

Huang Jie, Huang Jie. An Intelligent Tracking Control Method for Unmanned Vehicles with Time-varying Disturbances[J]. Journal of System Simulation, 2025, 37(4): 1063-1075.

Figures/Tables 12

Fig. 1

Fig. 2

Table 1

Model parameter setting of unmanned vehicles

参数	数值	参数	数值
$M$ /kg	6	$I w$ /(kg·m²)	0.094 5
$B$ /m	0.05	$a$ /(N/V)	0.087
$l a$ /m	0.11	$b$ /(kg/s)	11.4
$l b$ /m	0.18	$υ f x = υ f y$ /(m/s)	0.15

Table 1

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Table 2

References 27

1	张国辉, 王璇, 张雅楠, 等. 实际环境中多无人车协同路径规划模型研究[J]. 系统仿真学报, 2023, 35(2): 408-422.
	Zhang Guohui, Wang Xuan, Zhang Yanan, et al. Research on Cooperative Path Planning Model of Multiple Unmanned Vehicles in Real Environment[J]. Journal of System Simulation, 2023, 35(2): 408-422.
2	Kakkar S, Minor M A. Fast and Reliable Motion Model for Articulated Wheeled Mobile Robots on Extremely Rough and Rocky Terrains[J]. IEEE Robotics and Automation Letters, 2019, 4(3): 2252-2259.
3	Ren Chao, Li Xiaohan, Yang Xuebo, et al. Extended State Observer-based Sliding Mode Control of an Omnidirectional Mobile Robot with Friction Compensation[J]. IEEE Transactions on Industrial Electronics, 2019, 66(12): 9480-9489.
4	Li Jing, Wu Qingbin, Wang Junzheng, et al. Autonomous Tracking Control for Four-wheel Independent Steering Robot Based on Improved Pure Pursuit[J]. Journal of Beijing Institute of Technology, 2020, 29(4): 466-473.
5	张华强, 王国栋, 吕云飞, 等. 基于改进纯追踪模型的农机路径跟踪算法研究[J]. 农业机械学报, 2020, 51(9): 18-25.
	Zhang Huaqiqng, Wang Guodong, Yunfei Lü, et al. Agricultural Machinery Automatic Navigation Control System Based on Improved Pure Tracking Model[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(9): 18-25.
6	邓海鹏, 麻斌, 赵海光, 等. 自主驾驶车辆紧急避障的路径规划与轨迹跟踪控制[J]. 兵工学报, 2020, 41(3): 585-594.
	Deng Haipeng, Ma Bin, Zhao Haiguang, et al. Path Planning and Tracking Control of Autonomous Vehicle for Obstacle Avoidance[J]. Acta Armamentarii, 2020, 41(3): 585-594.
7	Kanchwala H, Ícaro Bezerra Viana, Ceccoti M, et al. Model Predictive Tracking Controller for a High Fidelity Vehicle Dynamics Model[C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Piscataway: IEEE, 2019: 1496-1503.
8	梁忠超, 张欢, 赵晶, 等. 基于自适应MPC的无人驾驶车辆轨迹跟踪控制[J]. 东北大学学报(自然科学版), 2020, 41(6): 835-840.
	Liang Zhongchao, Zhang Huan, Zhao Jing, et al. Trajectory Tracking Control of Unmanned Vehicles Based on Adaptive MPC[J]. Journal of Northeastern University(Natural Science), 2020, 41(6): 835-840.
9	陈亮, 秦兆博, 孔伟伟, 等. 基于最优前轮侧偏力的智能汽车LQR横向控制[J]. 清华大学学报(自然科学版), 2021, 61(9): 906-912.
	Chen Liang, Qin Zhaobo, Kong Weiwei, et al. Lateral Control Using LQR for Intelligent Vehicles Based on the Optimal Front-tire Lateral Force[J]. Journal of Tsinghua University(Science and Technology), 2021, 61(9): 906-912.
10	Wen Guoxing, Chen C L, Ge Shuzhi, et al. Optimized Adaptive Nonlinear Tracking Control Using Actor-critic Reinforcement Learning Strategy[J]. IEEE Transactions on Industrial Informatics, 2019, 15(9): 4969-4977.
11	Peng Zhinan, Luo Rui, Hu Jiangping, et al. Optimal Tracking Control of Nonlinear Multiagent Systems Using Internal Reinforce Q-learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(8): 4043-4055.
12	谭伟, 刘景升, 祖晖, 等. 参数不确定和扰动下智能汽车路径跟踪控制[J]. 浙江大学学报(工学版), 2023, 57(4): 702-711.
	Tan Wei, Liu Jingsheng, Zu Hui, et al. Intelligent Vehicle Path Tracking Control Under Parametric Uncertainties and External Disturbances[J]. Journal of Zhejiang University(Engineering Science), 2023, 57(4): 702-711.
13	Modares H, Lewis F L. Linear Quadratic Tracking Control of Partially-unknown Continuous-time Systems Using Reinforcement Learning[J]. IEEE Transactions on Automatic Control, 2014, 59(11): 3051-3056.
14	Gao Weinan, Jiang Zhongping. Learning-based Adaptive Optimal Tracking Control of Strict-feedback Nonlinear Systems[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(6): 2614-2624.
15	Chen Ci, Modares H, Xie Kan, et al. Reinforcement Learning-based Adaptive Optimal Exponential Tracking Control of Linear Systems with Unknown Dynamics[J]. IEEE Transactions on Automatic Control, 2019, 64(11): 4423-4438.
16	Liu Derong, Xue Shan, Zhao Bo, et al. Adaptive Dynamic Programming for Control: A Survey and Recent Advances[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2021, 51(1): 142-160.
17	Luo Rui, Peng Zhinan, Hu Jiangping, et al. Adaptive Optimal Control of Affine Nonlinear Systems via Identifier-critic Neural Network Approximation with Relaxed PE Conditions[J]. Neural Networks, 2023, 167: 588-600.
18	Jiang Yi, Gao Weinan, Wu Jin, et al. Reinforcement Learning and Cooperative H_∞ Output Regulation of Linear Continuous-time Multi-agent Systems[J]. Automatica, 2023, 148: 110768.
19	Zhao Jianguo, Yang Chunyu, Gao Weinan, et al. Linear Quadratic Tracking Control of Unknown Systems: A Two-phase Reinforcement Learning Method[J]. Automatica, 2023, 148: 110761.
20	Yang Yunjie, Wan Yan, Zhu Jihong, et al. H_∞Tracking Control for Linear Discrete-time Systems: Model-free Q-learning Designs[J]. IEEE Control Systems Letters, 2021, 5(1): 175-180.
21	于欣波, 贺威, 薛程谦, 等. 基于扰动观测器的机器人自适应神经网络跟踪控制研究[J]. 自动化学报, 2019, 45(7): 1307-1324.
	Yu Xinbo, He Wei, Xue Chengqian, et al. Disturbance Observer-based Adaptive Neural Network Tracking Control for Robots[J]. Acta Automatica Sinica, 2019, 45(7): 1307-1324.
22	Wang Hongbo, Hu Chenglei, Zhou Juntao, et al. Path Tracking Control of an Autonomous Vehicle with Model-free Adaptive Dynamic Programming and RBF Neural Network Disturbance Compensation[J]. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 2022, 236(5): 825-841.
23	Ma Qingwen, Zhang Xinglong, Xu Xin, et al. Self-learning Sliding Mode Control Based on Adaptive Dynamic Programming for Nonholonomic Mobile Robots[J]. ISA Transactions, 2023, 142: 136-147.
24	Alakshendra Veer, Chiddarwar Shital S. A Robust Adaptive Control of Mecanum Wheel Mobile Robot: Simulation and Experimental Validation[C]//2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway: IEEE, 2016: 5606-5611.
25	Wen Guoxing, Chen C L, Li Bin. Optimized Formation Control Using Simplified Reinforcement Learning for a Class of Multiagent Systems with Unknown Dynamics[J]. IEEE Transactions on Industrial Electronics, 2020, 67(9): 7879-7888.
26	吴克坚, 刘烁, 徐清华, 等. 应用Cauchy-schwarz不等式证明积分不等式举例[J]. 高等数学研究, 2021, 24(2): 13-15, 54.
	Wu Kejian, Liu Shuo, Xu Qinghua, et al. Using Cauchy-schwarz Inequality to Prove Integral Inequalities[J]. Studies in College Mathematics, 2021, 24(2): 13-15, 54.
27	王永军, 郑丽. 关于Young不等式和Schwarz不等式的证明[J]. 数学学习与研究, 2020(26): 145-147.

指标	本文方法	滑模控制	RL
代价函数平均值	717.396 1	924.490 7	1 802.500 0
跟踪误差平均值/m	0.033 0	0.054 7	3.467 1
代价函数最大值	1 432.7	16 781.0	3 918.1
跟踪误差最大值/m	0.054 8	0.075 8	6.867 8

[1]	Cheng Lei, Wang Haoyou, Chen Xinjie. Adaptive Tracking Control for Omnidirectional Vehicle Based on Characteristic Modeling [J]. Journal of System Simulation, 2024, 36(10): 2277-2287.
[2]	Guo Minghao, Ji Peng, Huang Haiwei. Unmanned Vehicle Path Planning and Tracking Control Based on Improved Artificial Potential Field Method [J]. Journal of System Simulation, 2024, 36(10): 2423-2434.
[3]	Guohui Zhang, Xuan Wang, Yanan Zhang, Ang Gao. Research on Cooperative Path Planning Model of Multiple Unmanned Vehicles in Real Environment [J]. Journal of System Simulation, 2023, 35(2): 408-422.
[4]	Lu Hongguang, Zhao Shuen. Research on Intelligent Vehicle Trajectory Tracking Control Based on Robust Model Prediction [J]. Journal of System Simulation, 2022, 34(1): 153-162.
[5]	Leng Yao, Zhao Shuen. Explicit Model Predictive Control for Intelligent Vehicle Lateral Trajectory Tracking [J]. Journal of System Simulation, 2021, 33(5): 1177-1187.
[6]	Ma Lixin, Lü Mengyuan. Adaptive Real-Time Tracking Control System for Ammonia Desulphurization [J]. Journal of System Simulation, 2018, 30(5): 1838-1843.
[7]	Fan Yunsheng, Sun Xiaojie, Wang Guofeng, Guo Chen. Tracking Control Method of Autonomous Dynamic Collision Avoidance for Unmanned Surface Vehicle [J]. Journal of System Simulation, 2018, 30(10): 3781-3788.
[8]	Ma Fei, Wu Yunjie, Cheng Mingzhi. Non-singular Terminal Sliding Mode Control for Bank-to-Turn Missile Based on Active Disturbance Rejection Decoupling Controller [J]. Journal of System Simulation, 2017, 29(9): 2175-2181.
[9]	Wang Yunjian, Liu Xiaoyun, Zhang Rongzhi, Xue Nana. State Feedback Control of Hybrid Active Power Filter [J]. Journal of System Simulation, 2017, 29(6): 1284-1289.