系统仿真学报 ›› 2026, Vol. 38 ›› Issue (10): 2924-2938.doi: 10.16182/j.issn1004731x.joss.26-0218

• 智能优化算法与自主决策 • 上一篇    下一篇

多任务图强化学习的五轴多工位工艺规划

刘俊杰1, 徐坚磊2, 鞠立涛2, 胡燕海1, 张庆浩2, 刘强2   

  1. 1.宁波大学 机械工程与力学学院,浙江 宁波 315211
    2.航工智能科技集团有限公司,浙江 宁波 315048
  • 收稿日期:2026-03-16 修回日期:2026-04-30 出版日期:2026-10-31 发布日期:2026-10-20
  • 通讯作者: 胡燕海
  • 第一作者简介:刘俊杰(2001-),男,硕士生,研究方向为智能制造。
  • 基金资助:
    国家自然科学基金(51705263);“科创甬江2035”重点研发计划(2025Z042)

Five-axis Multi-station Process Planning Based on Multi-task Graph Reinforcement Learning

Liu Junjie1, Xu Jianlei2, Ju Litao2, Hu Yanhai1, Zhang Qinghao2, Liu Qiang2   

  1. 1.Faculty of Mechanical Engineering & Mechanics, Ningbo University, Ningbo 315211, China
    2.Hanggong Intelligent Technology Group Co. , Ltd. , Ningbo 315048, China
  • Received:2026-03-16 Revised:2026-04-30 Online:2026-10-31 Published:2026-10-20
  • Contact: Hu Yanhai

摘要:

针对五轴多特征加工中状态空间呈阶乘级扩展与多目标干涉难题,提出一种融合多任务图感知与物理掩码近端策略优化 (multi-task graph and MPPO, MTG-MPPO) 的分层智能规划架构。构建融合边界表示与加工方向矢量的属性邻接图,通过多任务网络输出概率软标签量化工艺多解性,实现解空间的前置拓扑剪枝;建立融合有向无环图掩码与密集奖励重塑的MDP模型,并引入基于策略分布的蒙特卡罗寻优机制,统筹切削、换刀与转台代价。仿真结果表明:MTG-MPPO有效抑制了空间无序穿梭,加工总耗时较启发式基准算法缩减12.33%与9.81%。该模型在毫秒级推演下能以最高39%的概率跳出局部势垒逼近更优解,验证了以虚拟推演置换物理机床耗时的算力置换范式,为数字孪生环境下的工艺决策提供了高鲁棒性路径。

关键词: 智能工艺规划, 五轴数控加工, 图神经网络, 深度强化学习, 多目标寻优, 虚拟仿真

Abstract:

A hierarchical intelligent planning architecture integrating multi-task graph and masked proximal policy optimization (MTG-MPPO) is proposed to address the challenges of factorial expansion of the state space and multi-objective interference in five-axis multi-feature machining. An attributed adjacency graph integrating boundary representation and machining direction vectors is constructed. Probabilistic soft labels are output through a multi-task network to quantify process flexibility and thus achieve prior topological pruning of solution space. An MDP model integrating directed acyclic graph masks and dense reward shaping is built. A Monte Carlo (MC) optimization mechanism based on policy distribution is introduced to coordinate cutting, tool-changing, and rotary-table costs. Simulation results show that MTG-MPPO effectively suppresses disorderly spatial shuttling, reducing the total machining time by 12.33% and 9.81% compared with heuristic baseline algorithms. Under millisecond-level inference, the model escapes local barriers with a maximum probability of 39% to approach better solutions. This validates the computational substitution paradigm that replaces physical machine tool running time with virtual inference, providing a highly robust path for process decision-making in digital twin environments.

Key words: intelligent process planning, five-axis CNC machining, GNN, DRL, multi-objective optimization, virtual simulation

中图分类号: