Five-axis Multi-station Process Planning Based on Multi-task Graph Reinforcement Learning

doi:10.16182/j.issn1004731x.joss.26-0218

Abstract

Abstract:

A hierarchical intelligent planning architecture integrating multi-task graph and masked proximal policy optimization (MTG-MPPO) is proposed to address the challenges of factorial expansion of the state space and multi-objective interference in five-axis multi-feature machining. An attributed adjacency graph integrating boundary representation and machining direction vectors is constructed. Probabilistic soft labels are output through a multi-task network to quantify process flexibility and thus achieve prior topological pruning of solution space. An MDP model integrating directed acyclic graph masks and dense reward shaping is built. A Monte Carlo (MC) optimization mechanism based on policy distribution is introduced to coordinate cutting, tool-changing, and rotary-table costs. Simulation results show that MTG-MPPO effectively suppresses disorderly spatial shuttling, reducing the total machining time by 12.33% and 9.81% compared with heuristic baseline algorithms. Under millisecond-level inference, the model escapes local barriers with a maximum probability of 39% to approach better solutions. This validates the computational substitution paradigm that replaces physical machine tool running time with virtual inference, providing a highly robust path for process decision-making in digital twin environments.

Key words: intelligent process planning, five-axis CNC machining, GNN, DRL, multi-objective optimization, virtual simulation

CLC Number:

TP391.9

Liu Junjie, Xu Jianlei, Ju Litao, Hu Yanhai, Zhang Qinghao, Liu Qiang. Five-axis Multi-station Process Planning Based on Multi-task Graph Reinforcement Learning[J]. Journal of System Simulation, 2026, 38(10): 2924-2938.

Figures/Tables 13

Fig. 1

Fig. 2

Table 1

Hyperparameter configuration of virtual machining simulation platform and MTG-MPPO algorithm

系统模块	参数名称与物理意义	设定数值
底层运动学标定	航空铝基准切削线速度 $v c$ /(m/min)，每齿进给量 $f z$ /(mm/z)	250, 0.08
	机床主轴额定快移速度 $v r a p i d$ /(mm/min)	15 000
	单次物理换刀伺服延迟常数 $λ t c$ /s	8
	单次工作台旋转/姿态切换延迟常数 $λ r o t$ /s	15
GNN感知层配置	隐空间特征降维维度 $D h i d d e n$	128
	多头图注意力机制并发头数 $N h e a d$	4
	Adam优化器初始学习率 $α G N N$	1×10^-3
	多任务损失权重平衡系数( $α$ ∶ $β$ ∶ $γ$ )	1∶0.5∶0.5
PPO决策层配置	远期奖励衰减折扣因子 $γ$	0.99
	广义优势估计(GAE）平滑参数	0.95
	策略网络更新裁剪阈值 $ε G N N$	0.2
	MC策略并发采样次数K	100

Table 1

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Table 2

Fig. 8

Table 3

Fig. 9

Table 4

References 15

[1]	Zhang Yong, Wu Tao, Wang Yongfei, et al. Adaptive Nonlinear-error Control Method for Five-axis Machining of Complex Structures[J]. International Journal of Mechanical Sciences, 2026, 313: 111322.
[2]	牛帅, 仝晓萌, 蔡茂林, 等. 面向智能制造的数控加工工艺重用关键技术:系统回顾[J]. 机械工程学报, 2025, 61(20): 301-317.
	Niu Shuai, Tong Xiaomeng, Cai Maolin, et al. Key Technologies for CNC Machining Process Reuse for Intelligent Manufacturing: A Systematic Review[J]. Journal of Mechanical Engineering, 2025, 61(20): 301-317.
[3]	乔安杰, 许建新, 张振明, 等. 融合知识与深度学习的工艺自动生成方法[J/OL]. 计算机集成制造系统. (2025-12-01) [2026-03-10]. .
	Qiao Anjie, Xu Jianxin, Zhang Zhenming, et al. Automatic Process Generation Method Based on Knowledge and Deep Learning[J/OL]. Computer Integrated Manufacturing Systems. (2025-12-01) [2026-03-10]. .
[4]	Wang Pengyu, You Youpeng, Liu Weichao, et al. Knowledge-guided Deep Reinforcement Learning Framework for Feature Machining Step Sequencing[J]. The International Journal of Advanced Manufacturing Technology, 2025, 140(9): 5617-5636.
[5]	Liu Xiaojun, Yi Hong, Ni Zhonghua. Application of Ant Colony Optimization Algorithm in Process Planning Optimization[J]. Journal of Intelligent Manufacturing, 2013, 24(1): 1-13.
[6]	Peng Hong, Wang Han, Chen Daojia. Optimization of Remanufacturing Process Routes Oriented Toward Eco-efficiency[J]. Frontiers of Mechanical Engineering, 2019, 14(4): 422-433.
[7]	Kang Mujin, Kim Gyungha, Lee Taemoon, et al. Selection and Sequencing of Machining Processes for Prismatic Parts Using Process Ontology Model[J]. International Journal of Precision Engineering and Manufacturing, 2016, 17(3): 387-394.
[8]	Wang Zhen, Zhang Shusheng, Zhang Hang, et al. Machining Feature Process Route Planning Based on a Graph Convolutional Neural Network[J]. Advanced Engineering Informatics, 2024, 59(C): 102249.
[9]	汤文俊, 王裴岩, 蔡东风, 等. 基于编码器-解码器的工艺过程生成方法[J]. 计算机集成制造系统, 2023, 29(11): 3656-3668.
	Tang Wenjun, Wang Peiyan, Cai Dongfeng, et al. Encoder-decoder Based Process Generation Method[J]. Computer Integrated Manufacturing Systems, 2023, 29(11): 3656-3668.
[10]	Zhang Chao, Zhou Guanghui, Hu Junsheng, et al. Deep Learning-enabled Intelligent Process Planning for Digital Twin Manufacturing Cell[J]. Knowledge-Based Systems, 2020, 191: 105247.
[11]	Wu Wenbo, Huang Zhengdong, Zeng Jiani, et al. A Fast Decision-making Method for Process Planning with Dynamic Machining Resources via Deep Reinforcement Learning[J]. Journal of Manufacturing Systems, 2021, 58: 392-411.
[12]	Xiao Qinge, Niu Ben, Xue Bing, et al. Graph Convolutional Reinforcement Learning for Advanced Energy-aware Process Planning[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(5): 2802-2814.
[13]	Sugisawa Yasutomo, Takasugi Keigo, Asakawa Naoki. Machining Sequence Learning via Inverse Reinforcement Learning[J]. Precision Engineering, 2022, 73: 477-487.
[14]	Zhang Hang, Wang Wenhu, Zhang Shusheng, et al. A Novel Method Based on Deep Reinforcement Learning for Machining Process Route Planning[J]. Robotics and Computer-Integrated Manufacturing, 2024, 86: 102688.
[15]	Colligan A R, Robinson T T, Nolan D C, et al. Hierarchical CADNet: Learning from B-reps for Machining Feature Recognition[J]. Computer-Aided Design, 2022, 147: 103226.

模型	案例I/案例II
模型	总时间/s	换刀次数	转台次数
Greedy	921.1/684.0	23/13	18/21
Layer-First	912.2/730.6	23/15	18/23
MTG-MPPO	799.6/616.9	21/13	12/18

模型	案例I/案例II
模型	总时间/s	换刀	转台
w/o GNN	1 546.7/648.8	25/16	15/19
w/o TAD	1 093.4/712.6	16/17	38/27
w/o MC Search	861.3/693.2	22/14	16/22
本文方法	799.6/616.9	21/13	12/18

统计学评价指标	案例I	案例II
启发式最短距离基准/s	912.1	684.0
极小值/s	799.6	616.9
极大值/s	1 029.6	777.3
分布平均值/s	926.4	701.0
分布中位数/s	923.4	707.6
下四分位数/s	892.7	678.8
上四分位数/s	960.8	718.7
优于基准算法概率/%	39	30