Abstract:

To address the theoretical challenges of exploration and exploitation trade-offs and uncertainty modeling in multi-objective reinforcement learning (MORL), this study developed a learning framework, MO-PAC, based on PAC-Bayes theory. By introducing a multi-objective stochastic Critic network and a dynamic preference mechanism, the framework extended the conventional A2C architecture, enabling adaptive and efficient approximation of complex Pareto fronts. Experimental results demonstrate that in multi-objective MuJoCo environments, MO-PAC outperforms baseline algorithms, achieving approximately 20% improvement in hypervolume and 60% increase in expected utility, while exhibiting superior convergence efficiency and robustness. It verifies both theoretical value and practical performance advantages in multi-objective decision-making. The MO-PAC framework overcomes the theoretical limitations of existing methods in dynamic trade-off and uncertainty modeling, providing a novel methodological foundation for advancing the MORL theoretical framework.

Key words: multi-objective optimization, multi-objective reinforcement learning, actor-critic algorithm, uncertainty, PAC-Bayes theory