Abstract:

To address issues such as fixed behavior patterns and insufficient adaptability in complex adversarial environments exhibited by traditional wargame agent decision-making models, this paper proposes a multi-agent reinforcement learning method based on suboptimal demonstrations (MARLSD). The proposed method integrates reward relabeling with a self-imitation learning mechanism, effectively improving the training efficiency of multi-agent reinforcement learning algorithms in environments with large state-action spaces and sparse rewards, even when only a small number of suboptimal demonstrations are available, while encouraging agents to explore better strategies. Experimental results show that, compared with baselines such as QMIX and MAGAIL, MARLSD significantly improves performance and training efficiency, adapts to various value-decomposition multi-agent reinforcement learning algorithms, and achieves strong results using only a small number of suboptimal demonstration trajectories.

Key words: suboptimal demonstration, sparse reward, self-imitation learning, wargame simulation, multi-agent reinforcement learning