Abstract:

The learning process and working mechanism of deep reinforcement learning methods such as DQN are not transparent, and their decision basis and reliability cannot be perceived, which makes the decisions made by the model highly questionable and greatly limits the application scenarios of deep reinforcement learning. To explain the decision-making mechanism of intelligent agents, this paper proposes a gradient based saliency map generation algorithm SMGG. It uses the gradient information of feature maps generated by high-level convolutional layers to calculate the importance of different feature maps. With the known structure and internal parameters of the model, starting from the last layer of the model, the weight of different feature maps relative to the saliency map is generated by calculating the gradient of feature maps; it classifies the importance of features in both positive and negative directions, and uses weights with positive influence to weight the features captured in the feature map, forming a positive interpretation of the current decision; it uses weights that have a negative impact on other categories to weight the features captured in the feature map, forming a reverse interpretation of the current decision. The saliency map of the decision is generated by the two together, and the basis for the intelligent agent's decision-making behavior is obtained. The effectiveness of this method has been demonstrated through experiments.

Key words: DRL, saliency map, interpretability, agent, gradient